-- Logs begin at Wed 2022-10-26 10:16:54 UTC, end at Wed 2022-10-26 11:20:10 UTC. -- Oct 26 10:49:04.924143 dv-cinder-862679-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 26 10:49:04.935225 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 26 10:49:04 2022] *** Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: nodename: dv-cinder-862679-1-iscsi Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: machine: x86_64 Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: clock source: unix Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: pcre jit disabled Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: detected number of CPU cores: 4 Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: current working directory: / Oct 26 10:49:04.937277 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: detected binary path: /usr/bin/uwsgi-core Oct 26 10:49:04.937852 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: your processes number limit is 47680 Oct 26 10:49:04.937852 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: your memory page size is 4096 bytes Oct 26 10:49:04.937852 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: detected max file descriptor number: 2048 Oct 26 10:49:04.937852 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: lock engine: pthread robust mutexes Oct 26 10:49:04.937852 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: thunder lock: enabled Oct 26 10:49:04.937852 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 26 10:49:04.937852 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 26 10:49:04.977374 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: Python main interpreter initialized at 0x559a6dcd19c0 Oct 26 10:49:04.977374 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: python threads support enabled Oct 26 10:49:04.977374 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: your server socket listen backlog is limited to 100 connections Oct 26 10:49:04.977374 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: your mercy for graceful operations on workers is 90 seconds Oct 26 10:49:04.977693 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: mapped 403077 bytes (393 KB) for 2 cores Oct 26 10:49:04.977788 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: *** Operational MODE: preforking *** Oct 26 10:49:04.978060 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: *** uWSGI is running in multiple interpreter mode *** Oct 26 10:49:04.978060 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: spawned uWSGI master process (pid: 81595) Oct 26 10:49:04.978252 dv-cinder-862679-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 26 10:49:04.979302 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: spawned uWSGI worker 1 (pid: 81607, cores: 1) Oct 26 10:49:04.979302 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: spawned uWSGI worker 2 (pid: 81608, cores: 1) Oct 26 10:49:04.979302 dv-cinder-862679-1-iscsi devstack@keystone.service[81595]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 26 10:49:09.329361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:49:09.329361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: from cryptography.utils import int_from_bytes Oct 26 10:49:09.330107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:49:09.330107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: from cryptography.utils import int_from_bytes Oct 26 10:49:09.332458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:49:09.332458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: from cryptography.utils import int_from_bytes Oct 26 10:49:09.332908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:49:09.332908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: from cryptography.utils import int_from_bytes Oct 26 10:49:09.379712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 26 10:49:09.379930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 26 10:49:09.380124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] command line args: [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 26 10:49:09.380399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 26 10:49:09.381125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ================================================================================ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 26 10:49:09.381455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] admin_token = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.381808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config_dir = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.382002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.382186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] config_source = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.382273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 26 10:49:09.382351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] debug = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.382426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 26 10:49:09.382556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.382814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] command line args: [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 26 10:49:09.382814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 26 10:49:09.382941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.382999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ================================================================================ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 26 10:49:09.383135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.383233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] admin_token = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.383357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.383470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config_dir = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.383613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.383720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.383859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config_source = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.383959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] debug = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_config_append = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_dir = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.384972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_config_append = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_dir = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.385910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.386024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.386155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.386228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.386349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.386419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.386515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.386638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.387993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.388984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] publish_errors = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.389094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.389212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.389331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.389442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.389558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.389668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.389827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] standard_threads = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] publish_errors = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.390941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_journal = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_json = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_stderr = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] use_syslog = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] standard_threads = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.391841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.392059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.392172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.392293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.392405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.392522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.392634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_journal = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.392634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_json = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.392850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.393035 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_stderr = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.393154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.393267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_syslog = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.393381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.393491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:49:09.393607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.393730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.393843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.393952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.394982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.395941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.396063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.396850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.396978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.397890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398348 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.398926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399044 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.399964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.400978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.401883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.402954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.403945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.404962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.405982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.406899 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.407950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.408997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.409895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410764 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.410985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.411104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.411217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.411327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.411439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.411555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.411677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.411879 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.412980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413786 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.413898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.414942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415392 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.415900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416715 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.416939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.417971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.external = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.418921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.password = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] auth.token = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.external = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.419945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.password = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.token = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.420982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421206 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.421900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.422932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.423890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.424899 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.425921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.426955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.427070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.427182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.427324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.427450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.427725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.427886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428864 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.428977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.429946 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.430977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.431971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.432894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.433946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.434987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435428 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.435855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.436009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.436113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.436243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.441429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.441542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.441622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.441742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.441817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.441889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.441971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442601 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.442891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.443509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.443657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.443737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.443825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.443907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.443981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.444990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.445956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.446944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.447931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448776 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.448992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449542 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.449979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.450947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.451068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.451192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.451319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.451436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.451572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.451677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.451874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452001 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.452915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.453950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.454883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.455919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.456891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.457900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.458013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.458126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.458239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.458360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.458475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.459327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.459451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.459645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.459926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.460973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.461946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462086 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.462983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463434 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.463956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.464897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465010 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465355 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.465926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.466946 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.467962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.468056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.475425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.475553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.475631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.475746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.475838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.475912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.475996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81608) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.476580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.477172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:49:09.477172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 26 10:49:09.498828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.499060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.499286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.500872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.501107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.501335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.508251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.510410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.538129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.538261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.538422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.538507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.611255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.611696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.682133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.682370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.682724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.682961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.840566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.840794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.841019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.841241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.841460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.841679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.855129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.855330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.855550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.855767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.856010 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.856226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.873171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.873358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.873587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.873815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.874038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.874254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.874468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.875858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.876093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.876315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.876534 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.876749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.876964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.877177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.885763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.888637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.899495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.899733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.899994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.900222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.900445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.900525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.900663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.900746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.900881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81608) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.900983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.901151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.901366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.901578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.901791 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:49:09.922045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.922156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.922255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:49:09.922369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:49:09.922597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.922682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.922796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:49:09.922877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:49:09.923071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.923153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.923334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.923415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.923592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.923875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.923972 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.924145 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.933716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.934002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.934158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.934282 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.934439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.934571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.934713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.934994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.941555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.941917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.944632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.944924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.945209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.945476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.945568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.945759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.945840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.946963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.947106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.947243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.947383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.947517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.947657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.948028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.948304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.970319 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.971634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.971768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.972082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.973102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.973391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.978496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.978595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.978788 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.978872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.979071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.979153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.985335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.985427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.985627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.985720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.989704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.990532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.990821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.991107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.991390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.991669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.991975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.992186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:09.992281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.992481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.992766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.993050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.993328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.993605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.993873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:09.994147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.007934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.008214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.009922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.010200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.012309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.014279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.015681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.016014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.017482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.017775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.020209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.022578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.023647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.025723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.026874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.027180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.027485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.027812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.028104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.028386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.028986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.029287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.029575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.029861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.030137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.030424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.044636 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.044941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.045211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.045476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.047161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.047456 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.047724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.048013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.052315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.054726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.056024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.056305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.056702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.057607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.059116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.059361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.059444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.059653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.060089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.060381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.060664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.062083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.062366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.062649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.062925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.063316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.071965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.072256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.072642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.072930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.073228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.073526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.073806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.074956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.075241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.075523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.075822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.076226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.076507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.076786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.089183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.092098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.092581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.094993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.095439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.096454 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.098056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.098305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.098389 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.098629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.100695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.102271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.102562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.102958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.105460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.106926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.109217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.110013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.110311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.110602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.110713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.110885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.113666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.114061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.114346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.114625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.119241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.119534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.119850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.123484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.123731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.124090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.127598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81608) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.127917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.128208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.129638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.129638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.129638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.129638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.129638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.130028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.130135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.130434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.130725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.130990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.131255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81608) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.131462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:49:10.131750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.132057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.132334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.132605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.132876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.133148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.133425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.133693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.133975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.134246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.134580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.134857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.135138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:49:10.164955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559a6dcd19c0 pid: 81608 (default app) Oct 26 10:49:10.168289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10849a-ea53-410b-898e-0e80c1fb5246 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:10.168585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10849a-ea53-410b-898e-0e80c1fb5246 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:10.168861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10849a-ea53-410b-898e-0e80c1fb5246 None None] PATH_INFO: `/v3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:10.170843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559a6dcd19c0 pid: 81607 (default app) Oct 26 10:49:10.173042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Wed Oct 26 10:49:10 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:18.163856 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d56fd-12b4-4574-ad07-b48f7db9154c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:18.164405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d56fd-12b4-4574-ad07-b48f7db9154c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:18.164405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d56fd-12b4-4574-ad07-b48f7db9154c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:18.169486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:49:18.174640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:18.175153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:18.175254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:18.245942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: INFO dbcounter [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] Registered counter for database keystone Oct 26 10:49:18.267991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG oslo_db.sqlalchemy.engines [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81608) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 26 10:49:18.275018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writer thread running {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 26 10:49:18.454045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG passlib.handlers.bcrypt [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81608) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 26 10:49:18.454429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG passlib.handlers.bcrypt [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81608) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 26 10:49:18.512559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:18.514215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:18.644964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 26 10:49:19.155281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d56fd-12b4-4574-ad07-b48f7db9154c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:19.226112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: INFO dbcounter [None req-b17d56fd-12b4-4574-ad07-b48f7db9154c None None] Registered counter for database keystone Oct 26 10:49:19.245448 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG oslo_db.sqlalchemy.engines [None req-b17d56fd-12b4-4574-ad07-b48f7db9154c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81607) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 26 10:49:19.251623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writer thread running {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 26 10:49:19.371533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d56fd-12b4-4574-ad07-b48f7db9154c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:19.375838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ec73-55dc-4b12-a6b2-a7293ae7eea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:19.376470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ec73-55dc-4b12-a6b2-a7293ae7eea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:19.376751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ec73-55dc-4b12-a6b2-a7293ae7eea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:19.377025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ec73-55dc-4b12-a6b2-a7293ae7eea5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:19.385669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4ec73-55dc-4b12-a6b2-a7293ae7eea5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:19.437036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING keystone.common.rbac_enforcer.enforcer [None req-42b4ec73-55dc-4b12-a6b2-a7293ae7eea5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 26 10:49:19.437392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4ec73-55dc-4b12-a6b2-a7293ae7eea5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:19.446871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Oct 26 10:49:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 10:49:19.452843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:19.469931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d45486-a559-4300-bacc-ee9a9be95866 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:19.474351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a198ff-758f-41e1-aa65-d8851e0795ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:19.475453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a198ff-758f-41e1-aa65-d8851e0795ab admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:19.475737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a198ff-758f-41e1-aa65-d8851e0795ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:19.476042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a198ff-758f-41e1-aa65-d8851e0795ab admin admin] PATH_INFO: `/v3/projects/b00b7a3900004d669310a34e206f2fc2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:19.480554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a198ff-758f-41e1-aa65-d8851e0795ab admin admin] RBAC: Authorizing `identity:get_project(project_id=b00b7a3900004d669310a34e206f2fc2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:19.530961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a6a198ff-758f-41e1-aa65-d8851e0795ab admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 26 10:49:19.531306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a198ff-758f-41e1-aa65-d8851e0795ab admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:19.535848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 26 10:49:19 2022] GET /identity/v3/projects/b00b7a3900004d669310a34e206f2fc2 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:21.668127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb1e05-50bc-4dc9-aefc-d23a91a047d7 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:21.668759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb1e05-50bc-4dc9-aefc-d23a91a047d7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:21.668759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb1e05-50bc-4dc9-aefc-d23a91a047d7 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:21.672769 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:49:21.678403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050341d-5b70-4268-a4db-985acb4e6b07 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:21.678813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050341d-5b70-4268-a4db-985acb4e6b07 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:21.678954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050341d-5b70-4268-a4db-985acb4e6b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:21.698599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8050341d-5b70-4268-a4db-985acb4e6b07 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:21.700154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8050341d-5b70-4268-a4db-985acb4e6b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:21.707933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 10:49:21.976282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb1e05-50bc-4dc9-aefc-d23a91a047d7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:21.988301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb1e05-50bc-4dc9-aefc-d23a91a047d7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:21.992500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90c582-f600-48d8-8b43-7c81be7ef64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:21.993090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90c582-f600-48d8-8b43-7c81be7ef64c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:21.993546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90c582-f600-48d8-8b43-7c81be7ef64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:21.993824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90c582-f600-48d8-8b43-7c81be7ef64c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:22.001422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90c582-f600-48d8-8b43-7c81be7ef64c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:22.003200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90c582-f600-48d8-8b43-7c81be7ef64c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:22.012727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Oct 26 10:49:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 10:49:22.018058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050341d-5b70-4268-a4db-985acb4e6b07 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:22.027646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050341d-5b70-4268-a4db-985acb4e6b07 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:22.032020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc0cbb-eb4c-42a0-91df-6fff5bd888a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:22.032649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc0cbb-eb4c-42a0-91df-6fff5bd888a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:22.032930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc0cbb-eb4c-42a0-91df-6fff5bd888a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:22.033204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc0cbb-eb4c-42a0-91df-6fff5bd888a8 admin admin] PATH_INFO: `/v3/users/36c1a5e80256403c902f09eda7736a68` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:22.034527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc0cbb-eb4c-42a0-91df-6fff5bd888a8 admin admin] RBAC: Authorizing `identity:get_user(user_id=36c1a5e80256403c902f09eda7736a68)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:22.036415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc0cbb-eb4c-42a0-91df-6fff5bd888a8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:22.037953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Wed Oct 26 10:49:22 2022] GET /identity/v3/users/36c1a5e80256403c902f09eda7736a68 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:24.236701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84ea85-87f8-4e02-9594-e82f56670aa4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:24.237232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84ea85-87f8-4e02-9594-e82f56670aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:24.237232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84ea85-87f8-4e02-9594-e82f56670aa4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:24.238575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:24.244086 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9a424-3847-49a7-83af-b8494bb7de56 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:24.244509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9a424-3847-49a7-83af-b8494bb7de56 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:24.244642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9a424-3847-49a7-83af-b8494bb7de56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:24.263479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-98f9a424-3847-49a7-83af-b8494bb7de56 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:24.264688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-98f9a424-3847-49a7-83af-b8494bb7de56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:24.308165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bfc16-b928-4f76-91c2-8480c5df0d7d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:24.308371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bfc16-b928-4f76-91c2-8480c5df0d7d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:24.308656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bfc16-b928-4f76-91c2-8480c5df0d7d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:24.309980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:49:24.315244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:24.315523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:24.315819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:24.351648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:24.358101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG passlib.handlers.bcrypt [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81607) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 26 10:49:24.358497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG passlib.handlers.bcrypt [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81607) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 26 10:49:24.408054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:24.409606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:24.507379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:24.625228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9a424-3847-49a7-83af-b8494bb7de56 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:24.650798 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:24.729555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9a424-3847-49a7-83af-b8494bb7de56 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:24.763456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:24.764133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:24.764413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:24.764685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:24.774723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:24.776089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:24.784934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:24.786018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:24.788826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:49:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:24.791696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:24.875936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:24.876420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f954b7-72af-4d8e-8f8e-8eedb2db81b9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:24.878821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:24.896358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7c68d-50b0-43cc-a265-402f8886c9f9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:24.913518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:24.914466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:24.915326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:24.915703 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:24.916748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:24.918194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:24.931644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:24.932282 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:24.932560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:24.932829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:24.933489 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:24.933790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:24.935174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:24.946685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:24.952215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:24.971567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.sql.core [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:49:24.971567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:49:24.971567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [parameters: {'id': '07699e8c57b24cf6baa2fbca7c718dd7', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 10:49:24.971567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81607) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:49:24.990726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 26 10:49:24.991528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application yield resource Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise self.value Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application session.commit() Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 10:49:24.992441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.flush() Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application compat.raise_( Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 10:49:24.994116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 10:49:24.995004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:49:24.995875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [parameters: {'id': '07699e8c57b24cf6baa2fbca7c718dd7', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 26 10:49:24.996798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:24.997741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 8/16] 10.222.0.20 () {60 vars in 1137 bytes} [Wed Oct 26 10:49:24 2022] POST /identity/v3/domains => generated 190 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:49:24.998807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:25.053430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef190b3-0854-4d4e-b5a6-ceb7b130ee6e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:25.088225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:25.088860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:25.089144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:25.089419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] PATH_INFO: `/v3/users/36c1a5e80256403c902f09eda7736a68` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:25.090572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] RBAC: Authorizing `identity:get_user(user_id=36c1a5e80256403c902f09eda7736a68)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:25.092064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:25.093636 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:24 2022] GET /identity/v3/users/36c1a5e80256403c902f09eda7736a68 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:25.099040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:25.102763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a93abb-76f0-43d9-b7fa-020f353bbf94 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:25.115003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:25.137936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:25.138736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:25.138997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:25.139266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:25.156849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:25.158349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:49:25.175217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:25.176174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:49:25.177104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:49:25.177104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:49:25.177104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:25.177104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:49:25.182961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:25.200770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf03688-3d29-42b6-a81f-30baa1af864e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:25.204843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:25.234003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:25.234637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:25.234915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:25.235185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:25.236322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:25.237760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:25.239296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:25.246529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:25.290357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a495e4-b068-454c-8c05-6d474b146aab None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:25.324521 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af875-5ee8-427d-9418-3fcebacf6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:25.325115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af875-5ee8-427d-9418-3fcebacf6c46 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:25.325402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af875-5ee8-427d-9418-3fcebacf6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:25.325676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af875-5ee8-427d-9418-3fcebacf6c46 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:25.326317 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09af875-5ee8-427d-9418-3fcebacf6c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:25.326623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09af875-5ee8-427d-9418-3fcebacf6c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:25.327942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:25.328044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09af875-5ee8-427d-9418-3fcebacf6c46 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:25.345582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:25.346169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7c304-9d98-487f-8003-44bc0b25e66a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:25.384924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc00491-cfd9-468f-86cf-98d39e4e8ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:25.385607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc00491-cfd9-468f-86cf-98d39e4e8ab5 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:25.385889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc00491-cfd9-468f-86cf-98d39e4e8ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:25.386164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc00491-cfd9-468f-86cf-98d39e4e8ab5 None admin] PATH_INFO: `/v3/domains/default/users/36c1a5e80256403c902f09eda7736a68/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:25.388683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc00491-cfd9-468f-86cf-98d39e4e8ab5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=36c1a5e80256403c902f09eda7736a68, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:25.390123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc00491-cfd9-468f-86cf-98d39e4e8ab5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:25.593650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 11/21] 10.222.0.20 () {58 vars in 1361 bytes} [Wed Oct 26 10:49:25 2022] PUT /identity/v3/domains/default/users/36c1a5e80256403c902f09eda7736a68/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:49:28.190612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249090ce-a9b0-4ad0-8516-1a2f3984bbf0 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.191157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249090ce-a9b0-4ad0-8516-1a2f3984bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.191157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249090ce-a9b0-4ad0-8516-1a2f3984bbf0 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.192909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:28.200128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c3eb-537f-4f90-867a-f1caae5b3229 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.200610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c3eb-537f-4f90-867a-f1caae5b3229 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.200705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c3eb-537f-4f90-867a-f1caae5b3229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.215543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27e14-e122-45a7-ac4e-c31439cf44fb None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.216199 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27e14-e122-45a7-ac4e-c31439cf44fb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.216674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27e14-e122-45a7-ac4e-c31439cf44fb None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.218201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 12/23] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:49:28.223687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52814e66-7986-4458-be46-36efda0f4edf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.224272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52814e66-7986-4458-be46-36efda0f4edf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.224819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52814e66-7986-4458-be46-36efda0f4edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.226655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e4a9c3eb-537f-4f90-867a-f1caae5b3229 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:28.231810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e4a9c3eb-537f-4f90-867a-f1caae5b3229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:28.253517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-52814e66-7986-4458-be46-36efda0f4edf None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:28.255225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-52814e66-7986-4458-be46-36efda0f4edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:28.377168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:28.380482 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e18f06-e43b-4b37-9eaa-3a8ddbf886e4 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.383338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e18f06-e43b-4b37-9eaa-3a8ddbf886e4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.383765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e18f06-e43b-4b37-9eaa-3a8ddbf886e4 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.397118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 13/25] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:28 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:28.401202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:28.402794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9632-c7c0-4d8e-af66-5953504e3408 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.403232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9632-c7c0-4d8e-af66-5953504e3408 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.403668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9632-c7c0-4d8e-af66-5953504e3408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.413752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60470a-592d-4aa8-a5ed-65f1f9bdd9c3 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.414224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60470a-592d-4aa8-a5ed-65f1f9bdd9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.414699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60470a-592d-4aa8-a5ed-65f1f9bdd9c3 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.416213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 14/27] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:28.425424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aab694-05c4-4c7d-9c91-6ad8a5b97792 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:28.425877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aab694-05c4-4c7d-9c91-6ad8a5b97792 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:28.426314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aab694-05c4-4c7d-9c91-6ad8a5b97792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:28.456494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-46bd9632-c7c0-4d8e-af66-5953504e3408 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:28.458141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-46bd9632-c7c0-4d8e-af66-5953504e3408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:28.465631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-02aab694-05c4-4c7d-9c91-6ad8a5b97792 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:28.466820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-02aab694-05c4-4c7d-9c91-6ad8a5b97792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:28.476109 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:28.573819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 15/28] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:28.598928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:28.604151 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 14/29] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:28.819986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aab694-05c4-4c7d-9c91-6ad8a5b97792 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:28.924890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9632-c7c0-4d8e-af66-5953504e3408 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:28.975471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aab694-05c4-4c7d-9c91-6ad8a5b97792 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.055644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.056440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.056719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.056989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.079233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:29.086994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:29.088222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:29.088955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:29.089538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 16/30] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:29.090073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:29.093098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:29.114940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9632-c7c0-4d8e-af66-5953504e3408 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.155346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.156438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.156911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.157370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.169059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:29.171062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:49:29.188537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:29.189421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:49:29.190209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:49:29.190209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:49:29.190209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:29.192059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 15/31] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:29.193049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:29.203546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0b08f-3abe-4177-a989-3d58136d060a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.238838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.239447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.239724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.240040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.240710 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:29.242097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:29.253446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:29.303490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4371-0ae0-4e31-9a59-229cb1f795ef None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.336890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.337731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.338165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.338578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.339379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:29.340934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:29.439687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 16/32] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 26 10:49:29 2022] POST /identity/v3/roles => generated 219 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:29.440777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:29.462832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 17/33] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 26 10:49:29 2022] POST /identity/v3/roles => generated 225 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:29.464221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:29.536143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:29.578052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241a093-34f5-433a-9138-434b1c6de596 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.583389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd62eb-3f7f-4e15-be56-564a6119d767 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.595464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:29.624413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430830ca-1871-4952-b489-32f6c1bc8123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.625071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430830ca-1871-4952-b489-32f6c1bc8123 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.625346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430830ca-1871-4952-b489-32f6c1bc8123 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.625627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430830ca-1871-4952-b489-32f6c1bc8123 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.626297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430830ca-1871-4952-b489-32f6c1bc8123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:29.626605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430830ca-1871-4952-b489-32f6c1bc8123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:29.627803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.628276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430830ca-1871-4952-b489-32f6c1bc8123 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:29.628867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.629316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.629753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.630733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:29.631290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:29.632999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:29.643071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 18/34] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:29.644660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc521c2-4e5d-43b3-a6d6-ed6d6180c9d2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.644959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc521c2-4e5d-43b3-a6d6-ed6d6180c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.645234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc521c2-4e5d-43b3-a6d6-ed6d6180c9d2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.646411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 17/35] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:29.651404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 19/36] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:29.652863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:29.653344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b120b-9c8b-4a6d-beeb-a3e024020968 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.653668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b120b-9c8b-4a6d-beeb-a3e024020968 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.653971 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b120b-9c8b-4a6d-beeb-a3e024020968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.702293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-566b120b-9c8b-4a6d-beeb-a3e024020968 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:29.702293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-566b120b-9c8b-4a6d-beeb-a3e024020968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:29.781277 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043407a-4801-4815-bbdb-41092fa79470 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.789645 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 20/37] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:29.791163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b120b-9c8b-4a6d-beeb-a3e024020968 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:29.819330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1b9e6-f396-48cb-a96c-b5ceaedad6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.820496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1b9e6-f396-48cb-a96c-b5ceaedad6bf None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.821114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1b9e6-f396-48cb-a96c-b5ceaedad6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.821529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1b9e6-f396-48cb-a96c-b5ceaedad6bf None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.822329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1b9e6-f396-48cb-a96c-b5ceaedad6bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:29.824199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1b9e6-f396-48cb-a96c-b5ceaedad6bf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:29.851167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:29.900475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b120b-9c8b-4a6d-beeb-a3e024020968 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:29.933505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:29.934139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.934469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.934685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] PATH_INFO: `/v3/users/36c1a5e80256403c902f09eda7736a68` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.936120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] RBAC: Authorizing `identity:get_user(user_id=36c1a5e80256403c902f09eda7736a68)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:29.936705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 18/38] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:49:29 2022] POST /identity/v3/projects => generated 301 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:29.937570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:29.943768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 21/39] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:29 2022] GET /identity/v3/users/36c1a5e80256403c902f09eda7736a68 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:29.945017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:29.945583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53b9c-2c78-46f8-ae9a-5f9a7b7fedb8 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.945875 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53b9c-2c78-46f8-ae9a-5f9a7b7fedb8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.946150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea53b9c-2c78-46f8-ae9a-5f9a7b7fedb8 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.947685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 19/40] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:29.954577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54368a-dfd4-45e0-8488-504e575a724b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:29.955398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54368a-dfd4-45e0-8488-504e575a724b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:29.956022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54368a-dfd4-45e0-8488-504e575a724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:29.982594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-da54368a-dfd4-45e0-8488-504e575a724b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:29.984383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-da54368a-dfd4-45e0-8488-504e575a724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:30.014696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:30.076414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0ca2-399d-406f-913a-e23ce1f866a0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:30.080988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 20/41] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:30.081974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54368a-dfd4-45e0-8488-504e575a724b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:30.107589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:30.132159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:30.133183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:30.133646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:30.134649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:30.136830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:30.138689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:30.141362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 22/42] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:30.147142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:30.277114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062aa4bf-053b-4b3b-8f1f-6b158a24d3f4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:30.311683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:30.312342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:30.312720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:30.312893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:30.313621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.domain.id=default, user.id=36c1a5e80256403c902f09eda7736a68) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:30.313931 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:30.315364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:30.325308 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 23/43] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Oct 26 10:49:30 2022] GET /identity/v3/role_assignments?user.id=36c1a5e80256403c902f09eda7736a68&scope.domain.id=default&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 530 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:30.331865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54368a-dfd4-45e0-8488-504e575a724b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:30.360853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:30.373340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:30.374150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:30.374364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:30.374612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:30.375365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:30.376912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:30.418022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:30.447322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 21/44] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 26 10:49:30 2022] POST /identity/v3/roles => generated 223 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:30.475877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb1a65-62f4-4359-b1b2-0796859148f3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:30.520121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7e50a-12af-4e6d-adeb-686f6529abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:30.520786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7e50a-12af-4e6d-adeb-686f6529abe0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:30.521069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7e50a-12af-4e6d-adeb-686f6529abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:30.521345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7e50a-12af-4e6d-adeb-686f6529abe0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:30.522520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7e50a-12af-4e6d-adeb-686f6529abe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:30.524589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7e50a-12af-4e6d-adeb-686f6529abe0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:30.526226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 24/45] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:30.532354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:30.554547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:30.645620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f519-d95f-4fd4-966a-10783c13ce59 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:30.681029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5651962-62c5-46c3-9ff2-8dfeaaf46748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:30.681653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5651962-62c5-46c3-9ff2-8dfeaaf46748 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:30.681943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5651962-62c5-46c3-9ff2-8dfeaaf46748 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:30.682277 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5651962-62c5-46c3-9ff2-8dfeaaf46748 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:30.682952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5651962-62c5-46c3-9ff2-8dfeaaf46748 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:30.684434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5651962-62c5-46c3-9ff2-8dfeaaf46748 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:30.772273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:49:30 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:32.913829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eafdb9-f161-4e35-bdca-30309263fec4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:32.914422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eafdb9-f161-4e35-bdca-30309263fec4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:32.914422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eafdb9-f161-4e35-bdca-30309263fec4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:32.915745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:32.920880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81c95-48ac-46ef-99f5-39299cb1ee3b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:32.921249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81c95-48ac-46ef-99f5-39299cb1ee3b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:32.921438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81c95-48ac-46ef-99f5-39299cb1ee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:32.939855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e3a81c95-48ac-46ef-99f5-39299cb1ee3b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:32.941054 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e3a81c95-48ac-46ef-99f5-39299cb1ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:33.001328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:33.006504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:33.279211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eafdb9-f161-4e35-bdca-30309263fec4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:33.380522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eafdb9-f161-4e35-bdca-30309263fec4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:33.398001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:33.414720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ef46d-b6ed-4289-ad14-32854081f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:33.415404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ef46d-b6ed-4289-ad14-32854081f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:33.415687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ef46d-b6ed-4289-ad14-32854081f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:33.415992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ef46d-b6ed-4289-ad14-32854081f0c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:33.417103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ef46d-b6ed-4289-ad14-32854081f0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:33.418553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ef46d-b6ed-4289-ad14-32854081f0c0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:33.420265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 26/49] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:33.425853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a81c95-48ac-46ef-99f5-39299cb1ee3b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:33.522027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a81c95-48ac-46ef-99f5-39299cb1ee3b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:33.555966 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001b67d-6df9-4993-aa3b-08a6efef3c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:33.556515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001b67d-6df9-4993-aa3b-08a6efef3c85 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:33.556794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001b67d-6df9-4993-aa3b-08a6efef3c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:33.557063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001b67d-6df9-4993-aa3b-08a6efef3c85 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:33.557701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001b67d-6df9-4993-aa3b-08a6efef3c85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:33.559157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001b67d-6df9-4993-aa3b-08a6efef3c85 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:33.649576 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 24/50] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Oct 26 10:49:33 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:35.790594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e43e4-8d3f-4a72-b2ba-4cf3a00873dc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:35.791146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e43e4-8d3f-4a72-b2ba-4cf3a00873dc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:35.791146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e43e4-8d3f-4a72-b2ba-4cf3a00873dc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:35.792534 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 27/51] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:35.797778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5bd0b-63ef-42dc-b75f-417d201562c0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:35.798144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5bd0b-63ef-42dc-b75f-417d201562c0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:35.798332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5bd0b-63ef-42dc-b75f-417d201562c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:35.816827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8db5bd0b-63ef-42dc-b75f-417d201562c0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:35.818013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8db5bd0b-63ef-42dc-b75f-417d201562c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:35.863263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:35.884345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 25/52] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:36.154633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e43e4-8d3f-4a72-b2ba-4cf3a00873dc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:36.255848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e43e4-8d3f-4a72-b2ba-4cf3a00873dc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:36.288503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae9121-5109-4bfd-9b0b-6b6f5bf78bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:36.289640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9121-5109-4bfd-9b0b-6b6f5bf78bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:36.290014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9121-5109-4bfd-9b0b-6b6f5bf78bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:36.290288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9121-5109-4bfd-9b0b-6b6f5bf78bd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:36.291382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae9121-5109-4bfd-9b0b-6b6f5bf78bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:36.292889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae9121-5109-4bfd-9b0b-6b6f5bf78bd2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:36.294424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 28/53] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:36.300030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5bd0b-63ef-42dc-b75f-417d201562c0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:36.396941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5bd0b-63ef-42dc-b75f-417d201562c0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:36.430730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c3415-3071-42fe-a7d3-9b08ecb931d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:36.431262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c3415-3071-42fe-a7d3-9b08ecb931d0 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:36.431546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c3415-3071-42fe-a7d3-9b08ecb931d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:36.431846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c3415-3071-42fe-a7d3-9b08ecb931d0 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:36.432504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c3415-3071-42fe-a7d3-9b08ecb931d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:36.433940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c3415-3071-42fe-a7d3-9b08ecb931d0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:36.534081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 26/54] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:49:36 2022] POST /identity/v3/users => generated 275 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:39.777328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d594ff-3644-4326-9eee-a84611f7f093 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:39.778339 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d594ff-3644-4326-9eee-a84611f7f093 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:39.778813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d594ff-3644-4326-9eee-a84611f7f093 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:39.780855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 29/55] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:39.786369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d304a3-7c0f-4d99-88d1-5b3c511c717d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:39.787153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d304a3-7c0f-4d99-88d1-5b3c511c717d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:39.787600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d304a3-7c0f-4d99-88d1-5b3c511c717d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:39.809297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a2d304a3-7c0f-4d99-88d1-5b3c511c717d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:39.810841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a2d304a3-7c0f-4d99-88d1-5b3c511c717d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:39.822975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388aae-8ebf-4d7a-99b1-309b0b7dc5e3 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:39.825976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388aae-8ebf-4d7a-99b1-309b0b7dc5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:39.825976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388aae-8ebf-4d7a-99b1-309b0b7dc5e3 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:39.825976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 30/56] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:39.830529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210c05f-a2ef-4b2c-b772-56aec5efadab None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:39.831183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210c05f-a2ef-4b2c-b772-56aec5efadab None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:39.831776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210c05f-a2ef-4b2c-b772-56aec5efadab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:39.856588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0210c05f-a2ef-4b2c-b772-56aec5efadab None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:39.860128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:39.861091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0210c05f-a2ef-4b2c-b772-56aec5efadab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:39.925905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 27/57] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:39.957428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:39.983961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 31/58] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:40.239562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d304a3-7c0f-4d99-88d1-5b3c511c717d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:40.337893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210c05f-a2ef-4b2c-b772-56aec5efadab None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:40.427089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d304a3-7c0f-4d99-88d1-5b3c511c717d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:40.427089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc34ca6-916e-442f-91f9-b2ef017af391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:40.427089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34ca6-916e-442f-91f9-b2ef017af391 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.427089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34ca6-916e-442f-91f9-b2ef017af391 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.427444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34ca6-916e-442f-91f9-b2ef017af391 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.440677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc34ca6-916e-442f-91f9-b2ef017af391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8cc34ca6-916e-442f-91f9-b2ef017af391 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:40.448046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:40.449032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:40.449852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:40.450461 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 28/59] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:40.455854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b417d6-d136-4582-b528-f4d9184f2e4e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.455854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b417d6-d136-4582-b528-f4d9184f2e4e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.455854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b417d6-d136-4582-b528-f4d9184f2e4e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.455854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 29/60] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:40.456562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f8980-27ca-4d9e-9b08-22769b696aa5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.456562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f8980-27ca-4d9e-9b08-22769b696aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.456847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f8980-27ca-4d9e-9b08-22769b696aa5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.458564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 30/61] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:40.459152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210c05f-a2ef-4b2c-b772-56aec5efadab None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:40.459828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f8980-27ca-4d9e-9b08-22769b696aa5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:40.492746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:40.576552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f8980-27ca-4d9e-9b08-22769b696aa5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:40.601195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883d7b4-c4ba-4e1c-a2e7-abe95d8f6ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:40.601951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883d7b4-c4ba-4e1c-a2e7-abe95d8f6ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.602253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883d7b4-c4ba-4e1c-a2e7-abe95d8f6ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.602524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883d7b4-c4ba-4e1c-a2e7-abe95d8f6ea5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.610462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b338ea-f5d8-4277-8d6c-6ddc6f995d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:40.611072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b338ea-f5d8-4277-8d6c-6ddc6f995d76 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.611411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b338ea-f5d8-4277-8d6c-6ddc6f995d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.611695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b338ea-f5d8-4277-8d6c-6ddc6f995d76 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.612395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b338ea-f5d8-4277-8d6c-6ddc6f995d76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:40.612704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b338ea-f5d8-4277-8d6c-6ddc6f995d76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:40.612860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883d7b4-c4ba-4e1c-a2e7-abe95d8f6ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:40.614112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b338ea-f5d8-4277-8d6c-6ddc6f995d76 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1883d7b4-c4ba-4e1c-a2e7-abe95d8f6ea5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:40.618580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:40.619410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:40.620194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 32/62] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:49:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:49:40.622260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea08d4-d15e-4ca1-a641-4cf8345e3995 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.622552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea08d4-d15e-4ca1-a641-4cf8345e3995 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.622826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea08d4-d15e-4ca1-a641-4cf8345e3995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.628017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 31/63] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:49:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:40.628017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886063be-eb1b-4aa5-a734-3b9f80492f62 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.628017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886063be-eb1b-4aa5-a734-3b9f80492f62 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.628017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886063be-eb1b-4aa5-a734-3b9f80492f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.642398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-71ea08d4-d15e-4ca1-a641-4cf8345e3995 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:40.643027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-886063be-eb1b-4aa5-a734-3b9f80492f62 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:40.643593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-71ea08d4-d15e-4ca1-a641-4cf8345e3995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:40.644231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-886063be-eb1b-4aa5-a734-3b9f80492f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:40.684298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:40.709887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 33/64] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:40.711753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b1f04-51ae-479a-923f-343c6cd32615 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.712063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b1f04-51ae-479a-923f-343c6cd32615 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.712336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b1f04-51ae-479a-923f-343c6cd32615 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.713569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 34/65] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:40.714266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b1f04-51ae-479a-923f-343c6cd32615 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:40.715611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 32/66] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:40.716700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886063be-eb1b-4aa5-a734-3b9f80492f62 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:40.743940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:40.836975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886063be-eb1b-4aa5-a734-3b9f80492f62 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:40.842614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b1f04-51ae-479a-923f-343c6cd32615 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:40.878956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:40.879495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928504ad-dd3d-4263-9671-01ceb4cb450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:40.879611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.879852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.880130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] PATH_INFO: `/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.880237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928504ad-dd3d-4263-9671-01ceb4cb450a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.880411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928504ad-dd3d-4263-9671-01ceb4cb450a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.880688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928504ad-dd3d-4263-9671-01ceb4cb450a None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.881411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928504ad-dd3d-4263-9671-01ceb4cb450a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:40.881719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928504ad-dd3d-4263-9671-01ceb4cb450a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:40.883219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928504ad-dd3d-4263-9671-01ceb4cb450a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:40.892408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 33/67] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:40.893739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbec5b-e936-4e80-b7e1-6a6ecd634e81 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:40.894033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbec5b-e936-4e80-b7e1-6a6ecd634e81 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:40.894317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbec5b-e936-4e80-b7e1-6a6ecd634e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:40.897008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] RBAC: Authorizing `identity:get_user(user_id=85cda2bc952e4d8a99a088ee03cdeaf3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:40.899267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:40.901283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 35/68] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:40 2022] GET /identity/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:40.902111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:40.917733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-53cbec5b-e936-4e80-b7e1-6a6ecd634e81 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:40.918933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-53cbec5b-e936-4e80-b7e1-6a6ecd634e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:40.936402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:40.992026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 34/69] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:40.993206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbec5b-e936-4e80-b7e1-6a6ecd634e81 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.010803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.013716 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64fd01-6c78-4ee7-b742-558d3877cc83 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.058118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.058761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.059037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.059321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] PATH_INFO: `/v3/users/36c1a5e80256403c902f09eda7736a68` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.060478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] RBAC: Authorizing `identity:get_user(user_id=36c1a5e80256403c902f09eda7736a68)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.061923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.063857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 36/70] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:40 2022] GET /identity/v3/users/36c1a5e80256403c902f09eda7736a68 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:41.064424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.110155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbec5b-e936-4e80-b7e1-6a6ecd634e81 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.146484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.147043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.147324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.147594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.148857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.150276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.156590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 35/71] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:40 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:41.157413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.171830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.174986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafeb41-587d-4d86-8308-b2f7f6251889 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.215955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.216976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.217552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.217880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.230106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:41.234992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:41.235899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:41.236735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 37/72] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:41.237379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.243316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.264742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeadd77-a397-4172-97bf-737b08b6bf02 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.299407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.300062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.300341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.300615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.311420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:41.316327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:41.317169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:41.317928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:41.319937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 36/73] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:41.321547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.342962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff0768-ac51-4fff-abf3-94fc01f75343 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.377762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.378394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.378675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.378950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.380121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.381557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.383992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 38/74] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:41.384799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.411660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.422354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad5478-b5e8-4edc-a4ce-5e5413f62bc7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.455994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.456641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.456921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.457195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/85cda2bc952e4d8a99a088ee03cdeaf3/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.459092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa6e65e21f64df995b975bfb4ebdbba, user_id=85cda2bc952e4d8a99a088ee03cdeaf3, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.460565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.482094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a38611-dcb4-4178-8798-8208d026b311 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.516072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.516676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.516965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.517245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.517900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:41.518208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.519586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.524455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.529300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 37/75] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:41 2022] PUT /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/85cda2bc952e4d8a99a088ee03cdeaf3/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:49:41.548427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.548580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 39/76] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:41.548580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.654533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.655592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac9ea5-7482-4f92-bc90-182a3bf364c2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.668219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c0cbe-0a9c-4e4c-9512-e67f2f580754 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.696167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.696994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.697288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.697895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.700661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.701409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.704022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 38/77] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:41.705620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.706107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.706390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.706661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.707327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:41.708141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.708742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.709231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.720473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 40/78] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:41.721863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.782914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.819584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e9e79-7cf3-4e0a-8eeb-a2d457461de2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.837913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bc9ff-89db-4740-8eaf-9a9020de0c82 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:41.855737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.856450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.856728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.857010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/36c1a5e80256403c902f09eda7736a68/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.858939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa6e65e21f64df995b975bfb4ebdbba, user_id=36c1a5e80256403c902f09eda7736a68, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.860447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.874960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:41.879865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:41.879865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:41.879865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] PATH_INFO: `/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:41.879865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] RBAC: Authorizing `identity:get_user(user_id=85cda2bc952e4d8a99a088ee03cdeaf3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:41.879865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:41.880417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 41/79] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:41.881165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:41.908358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:41.939845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 39/80] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:41 2022] PUT /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/36c1a5e80256403c902f09eda7736a68/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:49:41.939845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:42.008875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998e189-f417-48e0-9046-f2872ffc839a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:42.054140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:42.056241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:42.056738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:42.057483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:42.058341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:42.060046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:42.069679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda57fe6-7f64-4603-b8fa-0b0e6b52310c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:42.100632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:42.110403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:42.111062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:42.111346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:42.111626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] PATH_INFO: `/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:42.112859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] RBAC: Authorizing `identity:get_user(user_id=85cda2bc952e4d8a99a088ee03cdeaf3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:42.114326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:42.116210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 40/81] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:41 2022] GET /identity/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:42.116773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:42.167081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 42/82] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:49:41 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:42.167703 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:42.239552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e58c2-6755-4273-9ada-2f77b8a27f6b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:42.254898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:42.281339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:42.281567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f3911-a80c-488c-8385-e89dd822ca45 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:42.282517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:42.283012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:42.283483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] PATH_INFO: `/v3/projects/91a8b89deee9479da2dffa23febbbcd9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:42.284974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] RBAC: Authorizing `identity:get_project(project_id=91a8b89deee9479da2dffa23febbbcd9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:42.286631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:42.288568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 41/83] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:42 2022] GET /identity/v3/projects/91a8b89deee9479da2dffa23febbbcd9 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:42.297198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:42.335673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:42.336423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:42.336705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:42.336983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:42.338416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:42.340992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:42.344279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 43/84] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:42 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:42.349273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:42.383665 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:42.463812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963cb82-a525-41ee-aaea-294f758cd8ee None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:42.504123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dbd4d-fe83-40af-9aac-ddc7849842ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:42.504870 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dbd4d-fe83-40af-9aac-ddc7849842ae None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:42.505163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dbd4d-fe83-40af-9aac-ddc7849842ae None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:42.505447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dbd4d-fe83-40af-9aac-ddc7849842ae None admin] PATH_INFO: `/v3/projects/91a8b89deee9479da2dffa23febbbcd9/users/85cda2bc952e4d8a99a088ee03cdeaf3/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:42.508306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dbd4d-fe83-40af-9aac-ddc7849842ae None admin] RBAC: Authorizing `identity:create_grant(project_id=91a8b89deee9479da2dffa23febbbcd9, user_id=85cda2bc952e4d8a99a088ee03cdeaf3, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:42.510280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dbd4d-fe83-40af-9aac-ddc7849842ae None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:42.529545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342b72-e945-42cb-9314-f8adc29b33e0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:42.569570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a18aa-8ec6-4489-9e52-998a4cea0f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:42.570594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a18aa-8ec6-4489-9e52-998a4cea0f68 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:42.570893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a18aa-8ec6-4489-9e52-998a4cea0f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:42.571174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a18aa-8ec6-4489-9e52-998a4cea0f68 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/85cda2bc952e4d8a99a088ee03cdeaf3/roles/a2f657dd4fab43288cb9e84eb6d94d51` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:42.573434 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a18aa-8ec6-4489-9e52-998a4cea0f68 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa6e65e21f64df995b975bfb4ebdbba, user_id=85cda2bc952e4d8a99a088ee03cdeaf3, role_id=a2f657dd4fab43288cb9e84eb6d94d51)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:42.574873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a18aa-8ec6-4489-9e52-998a4cea0f68 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:42.601199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 42/85] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:42 2022] PUT /identity/v3/projects/91a8b89deee9479da2dffa23febbbcd9/users/85cda2bc952e4d8a99a088ee03cdeaf3/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:49:42.660623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 44/86] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:42 2022] PUT /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/85cda2bc952e4d8a99a088ee03cdeaf3/roles/a2f657dd4fab43288cb9e84eb6d94d51 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:49:44.120196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b4cb0-2a54-4b16-b8d2-07beb3099d20 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:44.120720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b4cb0-2a54-4b16-b8d2-07beb3099d20 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:44.120807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b4cb0-2a54-4b16-b8d2-07beb3099d20 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:44.122467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 43/87] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:44.127940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28bb31-9b36-40dc-9b29-7a4857661285 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:44.128360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28bb31-9b36-40dc-9b29-7a4857661285 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:44.128541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28bb31-9b36-40dc-9b29-7a4857661285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:44.141011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:44.152761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2c28bb31-9b36-40dc-9b29-7a4857661285 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:44.158425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2c28bb31-9b36-40dc-9b29-7a4857661285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:44.248390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 45/88] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:44.789472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b4cb0-2a54-4b16-b8d2-07beb3099d20 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:44.909439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b4cb0-2a54-4b16-b8d2-07beb3099d20 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:44.935344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:44.945525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ede1f9-45b8-40c4-90ac-be081393facc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:44.953074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ede1f9-45b8-40c4-90ac-be081393facc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:44.953074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ede1f9-45b8-40c4-90ac-be081393facc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:44.953074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 46/89] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:44.955724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:44.956600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:44.957066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c754c8-03b9-4c17-8963-98ac58155822 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:44.957275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:44.957368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c754c8-03b9-4c17-8963-98ac58155822 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:44.957628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c754c8-03b9-4c17-8963-98ac58155822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:44.958019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:44.981906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:44.995644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:44.996534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:44.997304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 44/90] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:45.000928 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b3c754c8-03b9-4c17-8963-98ac58155822 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:45.002210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b3c754c8-03b9-4c17-8963-98ac58155822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:45.004558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:45.179540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 47/91] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:45.197945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791ebab-1148-4e33-8edc-53bea4dbd647 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:45.268514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:45.269146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:45.269431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:45.269710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:45.270385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:45.270693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:45.272129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:45.281189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 45/92] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:45.287132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c754c8-03b9-4c17-8963-98ac58155822 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:45.405631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:45.475715 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c754c8-03b9-4c17-8963-98ac58155822 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:45.525598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4117ba-bd17-4e32-aea7-4f094a3885c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:45.526251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4117ba-bd17-4e32-aea7-4f094a3885c9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:45.526538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4117ba-bd17-4e32-aea7-4f094a3885c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:45.526872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4117ba-bd17-4e32-aea7-4f094a3885c9 None admin] PATH_INFO: `/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:45.528461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4117ba-bd17-4e32-aea7-4f094a3885c9 None admin] RBAC: Authorizing `identity:get_user(user_id=85cda2bc952e4d8a99a088ee03cdeaf3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:45.530008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4117ba-bd17-4e32-aea7-4f094a3885c9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:45.531636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 48/93] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:45 2022] GET /identity/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:45.540924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:45.562108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a1f4-67ce-407f-a33a-a77aa870ac0d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:45.562404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a1f4-67ce-407f-a33a-a77aa870ac0d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:45.562676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a1f4-67ce-407f-a33a-a77aa870ac0d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:45.564366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 49/94] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:45.569734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46109c3-c838-464e-a4f1-4efcc426d6ce None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:45.570111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46109c3-c838-464e-a4f1-4efcc426d6ce None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:45.570355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46109c3-c838-464e-a4f1-4efcc426d6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:45.600158 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:45.603540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d46109c3-c838-464e-a4f1-4efcc426d6ce None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:45.606229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d46109c3-c838-464e-a4f1-4efcc426d6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:45.689922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3b3db-ebff-4382-a5c2-ad16c7d771a9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:45.743813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 50/95] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:45.745440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46109c3-c838-464e-a4f1-4efcc426d6ce None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:45.762785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b985d-2e10-477d-942b-8af8a9904b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:45.763516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b985d-2e10-477d-942b-8af8a9904b4d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:45.763826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b985d-2e10-477d-942b-8af8a9904b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:45.764159 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b985d-2e10-477d-942b-8af8a9904b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:45.836327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:45.889911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46109c3-c838-464e-a4f1-4efcc426d6ce None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:45.920572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b985d-2e10-477d-942b-8af8a9904b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:45.922068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b985d-2e10-477d-942b-8af8a9904b4d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:45.928130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80b1e1-9bc7-4912-80d3-65a39cdd0321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:45.929134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80b1e1-9bc7-4912-80d3-65a39cdd0321 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:45.929689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80b1e1-9bc7-4912-80d3-65a39cdd0321 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:45.930228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80b1e1-9bc7-4912-80d3-65a39cdd0321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:45.942572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80b1e1-9bc7-4912-80d3-65a39cdd0321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cf80b1e1-9bc7-4912-80d3-65a39cdd0321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:45.951326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:45.952711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:45.953479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:45.956181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 51/96] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:49:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:45.959215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2065aa4-5060-42b4-9756-a1ce3238850f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:45.959893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2065aa4-5060-42b4-9756-a1ce3238850f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:45.960369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2065aa4-5060-42b4-9756-a1ce3238850f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:45.962774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 52/97] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:45.964264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2065aa4-5060-42b4-9756-a1ce3238850f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.037821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:46.113579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 46/98] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:49:46.114824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7997d-e11c-4e3f-b9f2-98a27b8da8e1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.115111 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7997d-e11c-4e3f-b9f2-98a27b8da8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.115383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7997d-e11c-4e3f-b9f2-98a27b8da8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.130371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2065aa4-5060-42b4-9756-a1ce3238850f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.134559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ccc7997d-e11c-4e3f-b9f2-98a27b8da8e1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:46.135759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ccc7997d-e11c-4e3f-b9f2-98a27b8da8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:46.156393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:46.167608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40c4e3-988c-4c2f-9c44-81abefdfe38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.168246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c4e3-988c-4c2f-9c44-81abefdfe38d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.168528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c4e3-988c-4c2f-9c44-81abefdfe38d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.168804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c4e3-988c-4c2f-9c44-81abefdfe38d None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.169463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40c4e3-988c-4c2f-9c44-81abefdfe38d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:46.169769 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40c4e3-988c-4c2f-9c44-81abefdfe38d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:46.171162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40c4e3-988c-4c2f-9c44-81abefdfe38d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:46.178969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 53/99] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:46.180203 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6aa03-5a07-4f8a-a977-7dca91ace0a2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.180764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6aa03-5a07-4f8a-a977-7dca91ace0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.180764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6aa03-5a07-4f8a-a977-7dca91ace0a2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.182151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 54/100] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:46.182648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6aa03-5a07-4f8a-a977-7dca91ace0a2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.208970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 47/101] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:46.208970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45610f-8598-4d37-8c24-84fd6fc1048d None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.208970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45610f-8598-4d37-8c24-84fd6fc1048d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.208970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45610f-8598-4d37-8c24-84fd6fc1048d None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.212903 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 48/102] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:46.213616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45610f-8598-4d37-8c24-84fd6fc1048d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.292005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:46.305360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6aa03-5a07-4f8a-a977-7dca91ace0a2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.318409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45610f-8598-4d37-8c24-84fd6fc1048d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.342099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60eb65-0ba9-4a88-a6c2-ecb728fac503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.342751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60eb65-0ba9-4a88-a6c2-ecb728fac503 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.343019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60eb65-0ba9-4a88-a6c2-ecb728fac503 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.343295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60eb65-0ba9-4a88-a6c2-ecb728fac503 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.354890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60eb65-0ba9-4a88-a6c2-ecb728fac503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:46.358454 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.359080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.359365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.359650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-6f60eb65-0ba9-4a88-a6c2-ecb728fac503 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:46.360233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:46.361050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:49:46.361834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:46.362973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:46.362973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:46.369028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 49/103] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:46.369028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.369223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 55/104] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:46.369223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd58584-e119-4e4f-a5a2-335f18bc2b5a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.369223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd58584-e119-4e4f-a5a2-335f18bc2b5a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.369223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd58584-e119-4e4f-a5a2-335f18bc2b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.386188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ecd58584-e119-4e4f-a5a2-335f18bc2b5a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:46.387655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ecd58584-e119-4e4f-a5a2-335f18bc2b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:46.429785 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:46.459400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 56/105] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:46.461342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88552-1ee9-4b4e-973d-b789616256cf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.461631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88552-1ee9-4b4e-973d-b789616256cf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.461911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88552-1ee9-4b4e-973d-b789616256cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.481959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-09c88552-1ee9-4b4e-973d-b789616256cf None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:46.483164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-09c88552-1ee9-4b4e-973d-b789616256cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:46.490702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8dec6-5a81-4793-b77b-13459b25d376 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.527036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.527666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.527978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.528258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] PATH_INFO: `/v3/users/36c1a5e80256403c902f09eda7736a68` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.529415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] RBAC: Authorizing `identity:get_user(user_id=36c1a5e80256403c902f09eda7736a68)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:46.530875 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:46.537243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 50/106] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/users/36c1a5e80256403c902f09eda7736a68 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:46.537243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.545478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:46.561060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 57/107] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:46.564742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c88552-1ee9-4b4e-973d-b789616256cf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.649945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd481524-ed1c-4ee1-a723-a44e2ff6aed5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.687636 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.688514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.688974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.689462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.690425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:46.690893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:46.692455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:46.692898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c88552-1ee9-4b4e-973d-b789616256cf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.697976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:46.702970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 51/108] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:46.708703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.741855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.743324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.743616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.744197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.744939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] RBAC: Adding query filter params (role.id=b109a364aa914a98aee2e78986065def, scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba, user.id=85cda2bc952e4d8a99a088ee03cdeaf3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:46.745259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:46.748075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:46.754520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 58/109] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/role_assignments?user.id=85cda2bc952e4d8a99a088ee03cdeaf3&scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba&role.id=b109a364aa914a98aee2e78986065def => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:46.755363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.788389 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:46.832176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e531f9-3855-43cc-9213-fd7a6771c258 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.874337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.874990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.875274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.875552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.889103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:46.895232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:46.896231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:46.897024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 52/110] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:46.897755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:46.909253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b6c6e-78ef-4ad4-b02c-f56ba0625408 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:46.947827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:46.948242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:46.948526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:46.948800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:46.988623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.014987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de348f7-fd6c-4214-93e5-12df55537806 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.021689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.024157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.050195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.056004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.056659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.056984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.057333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] PATH_INFO: `/v3/users/87d17922b45c4305813a0904e29a94c2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.071420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] RBAC: Authorizing `identity:get_user(user_id=87d17922b45c4305813a0904e29a94c2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.072986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.074681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 53/111] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/users/87d17922b45c4305813a0904e29a94c2 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.075468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.130379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 59/112] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:49:47.131316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.180894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c3a50-b562-4e11-92f7-fca01aa1278f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.203637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.217223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.223866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.223866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.223866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.229332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:47.234211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:47.235234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 54/113] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:47.236097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.246306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f33a1-8f65-4447-b4ab-ab034504315e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.282278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.282910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.283355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.283635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.285142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.286598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.287353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.288285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 60/114] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.289165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.339917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b43f07-4cfe-4731-9f1a-7d504140f8eb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.373316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.373912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.374186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.374455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.375106 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:47.375402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.376821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.386006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 55/115] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.387015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.394704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa41156-da80-48cd-be8a-91bbf4786900 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.429061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.430815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.431097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.431658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.432818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.434246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.435871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 61/116] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.436627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.439848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.490981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9899-6127-465c-9675-f38d4c70f7c5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.503422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.525022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.526863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.526863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.526863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.527526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.529029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.530694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 56/117] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.531445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.539509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f135b7-1c2b-4e50-875c-8e5743c28d2d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.572957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.573565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.573852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.574130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.574781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:47.575089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.576491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.584338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 62/118] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.585127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.629683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838777f9-0eb8-4a01-bf4d-be44c2c2f2ba None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.649149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.664061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.664667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.664953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.665227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.665871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.667424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.684447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926cd32-63c9-46fd-bcab-d90a7eaf1742 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.716757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.717381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.717666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.717945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] PATH_INFO: `/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.719107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=85cda2bc952e4d8a99a088ee03cdeaf3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.720572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.723300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 63/119] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.723300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.752305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 57/120] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:49:47 2022] POST /identity/v3/projects => generated 302 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:47.757535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.814808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.833289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcd6ca-b69e-4225-a578-52df73b82ef7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.873877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.874565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.874850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.875131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/87d17922b45c4305813a0904e29a94c2/roles/60ffc030f91c44d89c115671baace7bd` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.877144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa6e65e21f64df995b975bfb4ebdbba, user_id=87d17922b45c4305813a0904e29a94c2, role_id=60ffc030f91c44d89c115671baace7bd)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.878842 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.881389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:47.891114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226adec9-2f2b-4e88-851f-0cc97d7251af None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:47.929460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:47.929460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:47.929460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:47.929460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:47.929990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba, user.id=36c1a5e80256403c902f09eda7736a68) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:47.930293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:47.931769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:47.939770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 58/121] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/role_assignments?user.id=36c1a5e80256403c902f09eda7736a68&scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:47.944313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:47.962245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 64/122] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:47 2022] PUT /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/users/87d17922b45c4305813a0904e29a94c2/roles/60ffc030f91c44d89c115671baace7bd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:49:47.963475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:48.098465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:48.102316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054e8c4-e3f8-43e1-bf08-2ebb99d7f83a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:48.108040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a140f-a3ee-4a4a-a224-4864f48e6681 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:48.161034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:48.161702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:48.162055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:48.162335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] PATH_INFO: `/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:48.163918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] RBAC: Authorizing `identity:get_user(user_id=85cda2bc952e4d8a99a088ee03cdeaf3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:48.164561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:48.165201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:48.165313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:48.165480 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:48.165758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:48.167452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 59/123] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/users/85cda2bc952e4d8a99a088ee03cdeaf3 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:48.174735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:48.258831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:48.267413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:48.268952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:48.308013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80bbed-7a81-49d1-a33a-b32e84204304 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:48.354089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:48.354706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:48.355083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:48.355358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:48.392102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:48.395318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 65/124] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:49:48.403564 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:48.443948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:48.445532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:48.530442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:48.540468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00981-74ca-4ea7-96f7-d9f075872e63 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:48.568915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 60/125] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:49:48.569866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:48.577614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:48.578516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:48.579018 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:48.579494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:48.580881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:48.582552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:48.584419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 66/126] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:48 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:48.595173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:48.647410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:48.698929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be562497-d73d-4c5c-911d-b8e22e5c1573 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:48.703403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21b0c0-4147-47d0-a3a6-7e3bb1085035 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:48.735306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:48.736002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:48.736283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:48.736664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:48.737501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] RBAC: Adding query filter params (role.id=a2f657dd4fab43288cb9e84eb6d94d51, scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba, user.id=85cda2bc952e4d8a99a088ee03cdeaf3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:48.737807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:48.739391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:48.747099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd7f16-b2d5-47a2-8c6d-fa3e983dea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:48.747747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd7f16-b2d5-47a2-8c6d-fa3e983dea73 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:48.748065 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd7f16-b2d5-47a2-8c6d-fa3e983dea73 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:48.749077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd7f16-b2d5-47a2-8c6d-fa3e983dea73 None admin] PATH_INFO: `/v3/projects/91a8b89deee9479da2dffa23febbbcd9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:48.750328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd7f16-b2d5-47a2-8c6d-fa3e983dea73 None admin] RBAC: Authorizing `identity:get_project(project_id=91a8b89deee9479da2dffa23febbbcd9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:48.751887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 67/127] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:49:48 2022] GET /identity/v3/role_assignments?user.id=85cda2bc952e4d8a99a088ee03cdeaf3&scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba&role.id=a2f657dd4fab43288cb9e84eb6d94d51 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:48.752151 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd7f16-b2d5-47a2-8c6d-fa3e983dea73 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:48.754068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 61/128] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:48 2022] GET /identity/v3/projects/91a8b89deee9479da2dffa23febbbcd9 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:48.760711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:48.792624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:48.881774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1b5d2-81c6-46fd-ab88-f7cdb028ce50 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:48.923987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665c5bc-f271-49df-83b4-b66fb1c54376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:48.924613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c5bc-f271-49df-83b4-b66fb1c54376 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:48.924897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c5bc-f271-49df-83b4-b66fb1c54376 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:48.925175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c5bc-f271-49df-83b4-b66fb1c54376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:48.925914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665c5bc-f271-49df-83b4-b66fb1c54376 None admin] RBAC: Adding query filter params (role.id=b109a364aa914a98aee2e78986065def, scope.project.id=91a8b89deee9479da2dffa23febbbcd9, user.id=85cda2bc952e4d8a99a088ee03cdeaf3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:48.926229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665c5bc-f271-49df-83b4-b66fb1c54376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:48.927696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665c5bc-f271-49df-83b4-b66fb1c54376 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:48.934985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 68/129] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:49:48 2022] GET /identity/v3/role_assignments?user.id=85cda2bc952e4d8a99a088ee03cdeaf3&scope.project.id=91a8b89deee9479da2dffa23febbbcd9&role.id=b109a364aa914a98aee2e78986065def => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:50.171380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2670-9503-434c-92ff-897d69acb8a7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2670-9503-434c-92ff-897d69acb8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.172024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2670-9503-434c-92ff-897d69acb8a7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.173326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 62/130] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:50.179136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cbf71-2d4c-445d-9970-ae8e63fc079a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.179613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cbf71-2d4c-445d-9970-ae8e63fc079a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.179691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cbf71-2d4c-445d-9970-ae8e63fc079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.199195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-495cbf71-2d4c-445d-9970-ae8e63fc079a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:50.200496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-495cbf71-2d4c-445d-9970-ae8e63fc079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:50.265436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98b0e1-ae53-44a0-b23d-c7b0de31ae92 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.265726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98b0e1-ae53-44a0-b23d-c7b0de31ae92 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.266010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98b0e1-ae53-44a0-b23d-c7b0de31ae92 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.267522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 63/131] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:49:50.272234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a749c-a126-4567-b9f5-53c89b46a5d9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.272519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a749c-a126-4567-b9f5-53c89b46a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.273348 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a749c-a126-4567-b9f5-53c89b46a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.277442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 69/132] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:50.291781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d47a749c-a126-4567-b9f5-53c89b46a5d9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:50.292980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d47a749c-a126-4567-b9f5-53c89b46a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:50.362369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 64/133] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:50.546075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cbf71-2d4c-445d-9970-ae8e63fc079a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:50.567218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:50.632923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a749c-a126-4567-b9f5-53c89b46a5d9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:50.642803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cbf71-2d4c-445d-9970-ae8e63fc079a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:50.676875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:50.677509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.677792 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.678065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.679756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:50.681290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:50.682872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 70/134] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:50.688204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:50.693167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:50.732105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a749c-a126-4567-b9f5-53c89b46a5d9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:50.767594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:50.768253 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.768539 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.768815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.775141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:50.779532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:50.784556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:50.789096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:50.789884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 65/135] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:49:50.790522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c26f5-1961-4059-b959-d14f5ba334b9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:50.792234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:50.823044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:50.823570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.823877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.824148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.824793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:50.826248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:50.891494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6cf85-a067-4a4e-a8bb-4e0b8c3c793d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:50.914700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:50.925890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:50.926490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:50.926789 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:50.927042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:50.927725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:50.928058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:50.929747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:50.937471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 66/136] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:49:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:50.942123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 71/137] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:49:50 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:49:50.945050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:51.054913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653736d4-08b4-4873-8a44-84a30819f6a3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:51.090259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:51.090918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:51.091209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:51.091490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] PATH_INFO: `/v3/users/87d17922b45c4305813a0904e29a94c2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:51.092727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] RBAC: Authorizing `identity:get_user(user_id=87d17922b45c4305813a0904e29a94c2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:51.094203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:51.095938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 67/138] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:50 2022] GET /identity/v3/users/87d17922b45c4305813a0904e29a94c2 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:51.102237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:51.205294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09399c30-9302-43b1-8a3d-c9b54abe2b17 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:51.240520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:51.241223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:51.241509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:51.241781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:51.256767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:51.313347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:51.314882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:51.427903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 72/139] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:49:51.434738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:51.495757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:51.545683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a30ae-c09d-4c49-9fc0-ee0bf0838c40 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:51.587907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194cff5-f0f0-40b8-b3a4-ff61923d6c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:51.588550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194cff5-f0f0-40b8-b3a4-ff61923d6c08 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:51.588834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194cff5-f0f0-40b8-b3a4-ff61923d6c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:51.589110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194cff5-f0f0-40b8-b3a4-ff61923d6c08 None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:51.590249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194cff5-f0f0-40b8-b3a4-ff61923d6c08 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:51.591702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194cff5-f0f0-40b8-b3a4-ff61923d6c08 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:51.593540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 68/140] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:51 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:51.599381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:51.650909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:51.708825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f95dd5-dd9d-498f-8ed7-429fcc3dc141 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:51.745492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b989b71-7ef7-404f-9140-fc9606864672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:51.746372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b989b71-7ef7-404f-9140-fc9606864672 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:51.746804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b989b71-7ef7-404f-9140-fc9606864672 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:51.747221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b989b71-7ef7-404f-9140-fc9606864672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:51.748124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b989b71-7ef7-404f-9140-fc9606864672 None admin] RBAC: Adding query filter params (role.id=60ffc030f91c44d89c115671baace7bd, scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba, user.id=87d17922b45c4305813a0904e29a94c2) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:51.748593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b989b71-7ef7-404f-9140-fc9606864672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:51.750259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b989b71-7ef7-404f-9140-fc9606864672 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:51.756860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 73/141] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:49:51 2022] GET /identity/v3/role_assignments?user.id=87d17922b45c4305813a0904e29a94c2&scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba&role.id=60ffc030f91c44d89c115671baace7bd => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:53.314984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b11f3-d449-4612-9110-36e08cb25eab None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.315486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b11f3-d449-4612-9110-36e08cb25eab None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.315486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b11f3-d449-4612-9110-36e08cb25eab None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.317018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 69/142] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:53.322435 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e680c31-d602-42a5-927f-89d9312dba27 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.322866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e680c31-d602-42a5-927f-89d9312dba27 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.323026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e680c31-d602-42a5-927f-89d9312dba27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.342189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6e680c31-d602-42a5-927f-89d9312dba27 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:53.343370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6e680c31-d602-42a5-927f-89d9312dba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:53.395983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06acfe0-0b02-4cde-a8db-6897583a941a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.396303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06acfe0-0b02-4cde-a8db-6897583a941a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.396700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06acfe0-0b02-4cde-a8db-6897583a941a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.399367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 70/143] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:49:53.405481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048f75b-7c6d-4702-8468-30f25e726056 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.405870 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048f75b-7c6d-4702-8468-30f25e726056 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.406152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048f75b-7c6d-4702-8468-30f25e726056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.419881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 74/144] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:53.427014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f048f75b-7c6d-4702-8468-30f25e726056 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:53.428235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f048f75b-7c6d-4702-8468-30f25e726056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:53.510761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 71/145] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:53.657376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448f08-0562-4213-a2c1-d174fd0aa901 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.657747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448f08-0562-4213-a2c1-d174fd0aa901 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.657937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448f08-0562-4213-a2c1-d174fd0aa901 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.659430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 75/146] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:53.664954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458206e-958b-4d89-8d0c-491264d404f3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.665246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458206e-958b-4d89-8d0c-491264d404f3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.665523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458206e-958b-4d89-8d0c-491264d404f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.683499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b458206e-958b-4d89-8d0c-491264d404f3 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:53.684703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b458206e-958b-4d89-8d0c-491264d404f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:53.690176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448f08-0562-4213-a2c1-d174fd0aa901 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:53.734199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:53.740674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:53.763831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 72/147] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:53.778626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458206e-958b-4d89-8d0c-491264d404f3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:53.803369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448f08-0562-4213-a2c1-d174fd0aa901 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:53.840361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:53.841294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.841905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.842328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.852931 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:53.858096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:53.859219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:53.860225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:53.862348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 76/148] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:49:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:49:53.869010 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:53.885217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458206e-958b-4d89-8d0c-491264d404f3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:53.920983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:53.921652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:53.921828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:53.922100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:53.933323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:53.938363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:53.939218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:49:53.944947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 73/149] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:49:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:49:53.945607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:53.973621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:53.978379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ad00-376c-4ab5-8770-21da79ece630 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:53.991091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:54.012942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.013554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.013837 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.014114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.014779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:54.015107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.016541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.025733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 77/150] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:54.028737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.053388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb255-6dfb-4e30-a23c-0ad741c116a7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.087551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.088199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.088475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.088741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.089398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:54.089697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.091072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.099528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 74/151] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:49:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:54.104141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.130035 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58981c3-a015-4de2-983e-f496ba4f0eac None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.164393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.165020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.165312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.165974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.166829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.168302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.170106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 78/152] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:54.170608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.199410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:54.200743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1e2c-32bd-4f3a-bfa3-24d232f49cce None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.202244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:54.234228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.234844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.235120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.235386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] PATH_INFO: `/v3/users/494ff4f835584035bb58d0613de05dfd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.248855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] RBAC: Authorizing `identity:get_user(user_id=494ff4f835584035bb58d0613de05dfd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.250298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.252075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 75/153] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:54 2022] GET /identity/v3/users/494ff4f835584035bb58d0613de05dfd => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:54.252825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.276768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e7c5-3e89-4006-87f0-d517089b0ad9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.310113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.310939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.311214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.311482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] PATH_INFO: `/v3/users/494ff4f835584035bb58d0613de05dfd` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.312615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] RBAC: Authorizing `identity:get_user(user_id=494ff4f835584035bb58d0613de05dfd)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.314044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.315934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 79/154] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:54 2022] GET /identity/v3/users/494ff4f835584035bb58d0613de05dfd => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:54.316604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.355597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ae472-db99-47d5-88c1-1d7ab8eafdeb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.388749 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.389338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.389613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.389882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.390518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.391969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.513854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:54.519339 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3028a-fa93-427d-a539-91595205c794 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.547596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 76/155] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:49:54 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:54.548467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.556654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.557354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.557660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.558009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.560307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.562275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.569334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 80/156] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:54 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:54.570527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.571720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:54.673354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e44543-960c-49cf-95bf-940e7e1b15ba None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.693799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ca74e-3376-4cb2-afe3-58682b0692dd None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.713438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.714081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.714356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.714666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.715772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.717273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.719186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 77/157] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:54 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:54.724883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:54.734219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43283789-9839-4489-a4c9-c4d85c3c014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.734891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283789-9839-4489-a4c9-c4d85c3c014f None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.735171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283789-9839-4489-a4c9-c4d85c3c014f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.735443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283789-9839-4489-a4c9-c4d85c3c014f None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb/users/494ff4f835584035bb58d0613de05dfd/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.737547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43283789-9839-4489-a4c9-c4d85c3c014f None admin] RBAC: Authorizing `identity:create_grant(project_id=72927d62c8b04432867a524b73620dcb, user_id=494ff4f835584035bb58d0613de05dfd, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.739040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43283789-9839-4489-a4c9-c4d85c3c014f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.813553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:54.830093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 81/158] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:54 2022] PUT /identity/v3/projects/72927d62c8b04432867a524b73620dcb/users/494ff4f835584035bb58d0613de05dfd/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:49:54.835994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d9d4-1582-4f69-acd0-10d692fe6c18 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:54.877798 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cafa3-7a28-43b4-9cca-170e67c55e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:54.878445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cafa3-7a28-43b4-9cca-170e67c55e7e None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:54.878722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cafa3-7a28-43b4-9cca-170e67c55e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:54.878991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cafa3-7a28-43b4-9cca-170e67c55e7e None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb/users/494ff4f835584035bb58d0613de05dfd/roles/a2f657dd4fab43288cb9e84eb6d94d51` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:54.882417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cafa3-7a28-43b4-9cca-170e67c55e7e None admin] RBAC: Authorizing `identity:create_grant(project_id=72927d62c8b04432867a524b73620dcb, user_id=494ff4f835584035bb58d0613de05dfd, role_id=a2f657dd4fab43288cb9e84eb6d94d51)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:54.883974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cafa3-7a28-43b4-9cca-170e67c55e7e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:54.957262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 78/159] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:54 2022] PUT /identity/v3/projects/72927d62c8b04432867a524b73620dcb/users/494ff4f835584035bb58d0613de05dfd/roles/a2f657dd4fab43288cb9e84eb6d94d51 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:49:56.871736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e29e24-579e-4403-a1ab-b225360a597b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:56.872330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e29e24-579e-4403-a1ab-b225360a597b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:56.872330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e29e24-579e-4403-a1ab-b225360a597b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:56.873888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 82/160] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:56.879080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a28c1-fa89-41e4-b123-7c4462d444c7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:56.879490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a28c1-fa89-41e4-b123-7c4462d444c7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:56.879631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a28c1-fa89-41e4-b123-7c4462d444c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:56.899304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-477a28c1-fa89-41e4-b123-7c4462d444c7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:56.900533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-477a28c1-fa89-41e4-b123-7c4462d444c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:56.945908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ae6d-f9b2-4430-886f-62ba50972ff4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:56.946216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ae6d-f9b2-4430-886f-62ba50972ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:56.946447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ae6d-f9b2-4430-886f-62ba50972ff4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:56.948103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 83/161] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:56.953989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec3d6-f783-48e5-bbcd-0b2205330a58 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:56.954268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec3d6-f783-48e5-bbcd-0b2205330a58 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:56.954529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec3d6-f783-48e5-bbcd-0b2205330a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:56.973091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-513ec3d6-f783-48e5-bbcd-0b2205330a58 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:56.974264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-513ec3d6-f783-48e5-bbcd-0b2205330a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:56.978728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 79/162] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:56.993001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:57.051860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 84/163] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:57.060740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63833f2-ead0-4920-bab7-cbb910655669 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.061179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63833f2-ead0-4920-bab7-cbb910655669 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.061562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63833f2-ead0-4920-bab7-cbb910655669 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.063050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 80/164] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:49:57.068683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234ba47-299a-4045-af8b-f5d7376a0055 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.069163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234ba47-299a-4045-af8b-f5d7376a0055 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.069517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234ba47-299a-4045-af8b-f5d7376a0055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.088113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b234ba47-299a-4045-af8b-f5d7376a0055 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:57.089299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b234ba47-299a-4045-af8b-f5d7376a0055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:57.156397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161731f2-99e3-464b-ab3f-1c376e2e2e96 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.156698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161731f2-99e3-464b-ab3f-1c376e2e2e96 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.156890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161731f2-99e3-464b-ab3f-1c376e2e2e96 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.158338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 81/165] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:49:57.166094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 85/166] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:49:57.166456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582182-0d2e-4820-a599-f030088dacbd None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.166921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582182-0d2e-4820-a599-f030088dacbd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.167366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582182-0d2e-4820-a599-f030088dacbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.186002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a9582182-0d2e-4820-a599-f030088dacbd None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:49:57.187444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a9582182-0d2e-4820-a599-f030088dacbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:49:57.258483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 82/167] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:49:57.302411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234ba47-299a-4045-af8b-f5d7376a0055 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.330866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9582182-0d2e-4820-a599-f030088dacbd None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.334353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:57.395635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:57.412775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234ba47-299a-4045-af8b-f5d7376a0055 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:57.436737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9582182-0d2e-4820-a599-f030088dacbd None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:57.450202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:57.451185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.451618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.452462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.465096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:57.470739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:57.471752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:49:57.472733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:57.474674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:57.474814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 86/168] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:57.477737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.477882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.477882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.477882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.478845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:57.480616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:57.482336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 83/169] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:57.482976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.559293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:57.587988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc36cfa-75ba-4017-8952-b1b60913fa30 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:57.590948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8d51e-e837-4434-8f57-96b74c32812b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:57.621998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:57.622604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.622881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.623152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.623832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:57.624135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:57.624837 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:57.625454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.625546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:57.625727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.625995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.634027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 84/170] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:57.634804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.637569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:57.642867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:57.643818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:57.644630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 87/171] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:57.645271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.649094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:57.738879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e5822-66be-41cc-bfe7-3c22329ddad0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:57.746809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926e549-6670-4455-8c2c-829ceab72cf4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:57.772157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:57.773000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.773155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.773431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.774074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:57.775520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:57.780522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:57.781145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.781421 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.781693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.783507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:57.791925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:49:57.797188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:49:57.798370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 88/172] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:49:57.799326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.858619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:57.862133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 85/173] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:49:57 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:49:57.862975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:57.900736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01060b3-9a38-4b08-9cd7-9e7f4f010fd6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:57.939233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:57.939899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:57.940176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:57.940444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] PATH_INFO: `/v3/users/1b5e6d49a24a459ea419d8d0e29078ff` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:57.957114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] RBAC: Authorizing `identity:get_user(user_id=1b5e6d49a24a459ea419d8d0e29078ff)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:57.958630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:57.961460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 89/174] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/users/1b5e6d49a24a459ea419d8d0e29078ff => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:57.962070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.000637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9f51-a63c-4291-87a5-f18bace2e2df None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.043455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.044350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.044781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.045052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.045718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:58.046021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.047403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.056613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 86/175] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:58.057446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.089384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f650c44-b149-4583-b535-3081bea12681 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.114397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:58.117970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:58.131883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284c299-e068-44e6-8824-6650b726480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.132960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284c299-e068-44e6-8824-6650b726480b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.132960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284c299-e068-44e6-8824-6650b726480b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.133127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284c299-e068-44e6-8824-6650b726480b None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.134188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284c299-e068-44e6-8824-6650b726480b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:58.134188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284c299-e068-44e6-8824-6650b726480b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.135372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284c299-e068-44e6-8824-6650b726480b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.146658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 90/176] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:49:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:58.147542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284c299-e068-44e6-8824-6650b726480b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.170462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea84d0-3f7a-4e5c-ae01-c9cbca7bc16c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.208702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.209469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.209734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.210074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.211221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.212712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.214655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 87/177] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:58 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:58.215826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.283180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284c299-e068-44e6-8824-6650b726480b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.320144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.320786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.321218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.321495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] PATH_INFO: `/v3/users/494ff4f835584035bb58d0613de05dfd` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.322659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] RBAC: Authorizing `identity:get_user(user_id=494ff4f835584035bb58d0613de05dfd)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.324155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.325851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 91/178] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:58 2022] GET /identity/v3/users/494ff4f835584035bb58d0613de05dfd => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:58.326615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.352003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807440f3-91b4-43d8-9b2c-303d70aaaaff None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.381586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:58.387493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:58.398355 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.399234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.399674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.400126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] PATH_INFO: `/v3/users/494ff4f835584035bb58d0613de05dfd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.401506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] RBAC: Authorizing `identity:get_user(user_id=494ff4f835584035bb58d0613de05dfd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.403136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.404940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 88/179] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:49:58 2022] GET /identity/v3/users/494ff4f835584035bb58d0613de05dfd => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:58.406013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.440560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d7ae3-7517-49d1-acac-01a0101314b8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.478490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.479313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.479691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.480036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb/users/1b5e6d49a24a459ea419d8d0e29078ff/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.482085 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] RBAC: Authorizing `identity:create_grant(project_id=72927d62c8b04432867a524b73620dcb, user_id=1b5e6d49a24a459ea419d8d0e29078ff, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.483618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.518922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23735e3a-7398-4d66-b472-331cd0d1f4c7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.555089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.555622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.555931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.556204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.558289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 92/180] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:49:58 2022] PUT /identity/v3/projects/72927d62c8b04432867a524b73620dcb/users/1b5e6d49a24a459ea419d8d0e29078ff/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:49:58.560763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.628792 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:58.643167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.644835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.718979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44939e38-8313-4d1d-816d-51c081d9e47c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:58.757824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:58.768143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 89/181] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:49:58.774636 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:58.795243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:58.795896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:58.796176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:58.796449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:58.929927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:58.932230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:58.933776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:58.941034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ae5e1-7432-4c30-a6b2-e4682b3c491d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:59.010060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:59.011737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:59.012254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:59.012735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:59.014087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:59.023809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:59.026040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 90/182] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:58 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:59.031312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:59.077913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 93/183] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:49:59.086745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:59.120387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:59.250764 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b3fb-c3f8-4433-b5be-1039d74f5588 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:59.270748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba581-ed6c-4f5a-beba-f35e995f90f1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:59.394919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:59.395535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:59.395826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:59.396098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:59.396890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=72927d62c8b04432867a524b73620dcb, user.id=494ff4f835584035bb58d0613de05dfd) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:59.397200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:59.399040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:59.410854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 91/184] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:49:59 2022] GET /identity/v3/role_assignments?user.id=494ff4f835584035bb58d0613de05dfd&scope.project.id=72927d62c8b04432867a524b73620dcb&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:49:59.420924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719b6e-045b-4564-a1e3-0b3d23ed89b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:59.421597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719b6e-045b-4564-a1e3-0b3d23ed89b2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:59.421875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719b6e-045b-4564-a1e3-0b3d23ed89b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:59.422147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719b6e-045b-4564-a1e3-0b3d23ed89b2 None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:59.423422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719b6e-045b-4564-a1e3-0b3d23ed89b2 None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:59.424966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719b6e-045b-4564-a1e3-0b3d23ed89b2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:59.426740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 94/185] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:49:59 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:49:59.434233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:49:59.464311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:49:59.574460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b3e6c-ef13-4157-ac32-0783b847ab9d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:49:59.624316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999514a8-99b2-48f5-a9fc-ea799f33c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:49:59.624931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999514a8-99b2-48f5-a9fc-ea799f33c3be None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:49:59.625210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999514a8-99b2-48f5-a9fc-ea799f33c3be None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:49:59.625480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999514a8-99b2-48f5-a9fc-ea799f33c3be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:49:59.626212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999514a8-99b2-48f5-a9fc-ea799f33c3be None admin] RBAC: Adding query filter params (role.id=a2f657dd4fab43288cb9e84eb6d94d51, scope.project.id=72927d62c8b04432867a524b73620dcb, user.id=494ff4f835584035bb58d0613de05dfd) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:49:59.626522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999514a8-99b2-48f5-a9fc-ea799f33c3be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:49:59.627983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999514a8-99b2-48f5-a9fc-ea799f33c3be None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:49:59.636463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 92/186] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:49:59 2022] GET /identity/v3/role_assignments?user.id=494ff4f835584035bb58d0613de05dfd&scope.project.id=72927d62c8b04432867a524b73620dcb&role.id=a2f657dd4fab43288cb9e84eb6d94d51 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:00.327628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d326e-5c19-487c-9890-98815c2fa1a1 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:00.328151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d326e-5c19-487c-9890-98815c2fa1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:00.328151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d326e-5c19-487c-9890-98815c2fa1a1 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:00.329595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 95/187] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:00.334527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aaea6-fd2a-462c-b7ea-61bfc8419125 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:00.334893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aaea6-fd2a-462c-b7ea-61bfc8419125 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:00.335123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aaea6-fd2a-462c-b7ea-61bfc8419125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:00.353634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-532aaea6-fd2a-462c-b7ea-61bfc8419125 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:00.354828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-532aaea6-fd2a-462c-b7ea-61bfc8419125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:00.421896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 93/188] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:00.591681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07604b-0524-4bab-9fa8-6d7a952eb320 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:00.592048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07604b-0524-4bab-9fa8-6d7a952eb320 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:00.592218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07604b-0524-4bab-9fa8-6d7a952eb320 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:00.593652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 96/189] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:00.598842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61a5a8-6f22-4b6d-a6b9-badb43550d4e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:00.599126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61a5a8-6f22-4b6d-a6b9-badb43550d4e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:00.599395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61a5a8-6f22-4b6d-a6b9-badb43550d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:00.617941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0d61a5a8-6f22-4b6d-a6b9-badb43550d4e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:00.619110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0d61a5a8-6f22-4b6d-a6b9-badb43550d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:00.628769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:00.686240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 94/190] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:00.689200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07604b-0524-4bab-9fa8-6d7a952eb320 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:00.774857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:00.794316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07604b-0524-4bab-9fa8-6d7a952eb320 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:00.828994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:00.829617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:00.829889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:00.830156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:00.840079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:00.844898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:00.845756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:00.846783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 97/191] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:00.851886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61a5a8-6f22-4b6d-a6b9-badb43550d4e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:00.957423 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61a5a8-6f22-4b6d-a6b9-badb43550d4e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:00.965867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:00.996341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e53618-60d5-465e-875f-f5faaba2bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:00.996968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53618-60d5-465e-875f-f5faaba2bba8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:00.997272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53618-60d5-465e-875f-f5faaba2bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:00.997549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53618-60d5-465e-875f-f5faaba2bba8 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:00.998212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e53618-60d5-465e-875f-f5faaba2bba8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:00.998522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e53618-60d5-465e-875f-f5faaba2bba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:00.999933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e53618-60d5-465e-875f-f5faaba2bba8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:01.008391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 95/192] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:01.009700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f970e-779d-4cad-aebb-1bbba05f7d38 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.009983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f970e-779d-4cad-aebb-1bbba05f7d38 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.010252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f970e-779d-4cad-aebb-1bbba05f7d38 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.011717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 96/193] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:01.013916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f970e-779d-4cad-aebb-1bbba05f7d38 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:01.023443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:01.078296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711292c-ae62-42dd-baa9-d0492b974632 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.110829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aead95-d2b5-4299-9271-997b29551e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.111447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aead95-d2b5-4299-9271-997b29551e8d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.111720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aead95-d2b5-4299-9271-997b29551e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.112024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aead95-d2b5-4299-9271-997b29551e8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.113106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aead95-d2b5-4299-9271-997b29551e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:01.114532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aead95-d2b5-4299-9271-997b29551e8d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:01.115464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f970e-779d-4cad-aebb-1bbba05f7d38 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.116782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 98/194] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:01.118035 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ac2f8-b4c2-4dbb-868f-2972839f5adf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.118314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ac2f8-b4c2-4dbb-868f-2972839f5adf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.118582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ac2f8-b4c2-4dbb-868f-2972839f5adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.135822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8b9ac2f8-b4c2-4dbb-868f-2972839f5adf None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:01.137276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8b9ac2f8-b4c2-4dbb-868f-2972839f5adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:01.139214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:01.154195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.154826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.155104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.155394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] PATH_INFO: `/v3/users/126d9e96a4194354828f2e39799c3427` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.168830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] RBAC: Authorizing `identity:get_user(user_id=126d9e96a4194354828f2e39799c3427)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:01.170289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:01.171935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 97/195] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:01 2022] GET /identity/v3/users/126d9e96a4194354828f2e39799c3427 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:01.172763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:01.208378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 99/196] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:01.209206 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ac2f8-b4c2-4dbb-868f-2972839f5adf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:01.234516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:01.272103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2053b-5ee3-457f-a5a5-bbcb92d52e2d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.309047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.309725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.310005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.310278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.310930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:01.312416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:01.317211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ac2f8-b4c2-4dbb-868f-2972839f5adf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.353460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.354088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.354347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.354612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.355707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:01.357150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:01.358888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 100/197] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:01 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:01.366748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:01.373117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:01.403815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 98/198] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:01 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:01.484417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54910d-b013-4eac-8db4-e26798d6e339 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.488796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:01.526647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.527347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.527612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.527925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb/users/126d9e96a4194354828f2e39799c3427/roles/60ffc030f91c44d89c115671baace7bd` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.529904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] RBAC: Authorizing `identity:create_grant(project_id=72927d62c8b04432867a524b73620dcb, user_id=126d9e96a4194354828f2e39799c3427, role_id=60ffc030f91c44d89c115671baace7bd)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:01.531368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:01.598592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e50ea-9a2a-482f-a3c0-b7b3780dc0ce None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.609739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 101/199] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:01 2022] PUT /identity/v3/projects/72927d62c8b04432867a524b73620dcb/users/126d9e96a4194354828f2e39799c3427/roles/60ffc030f91c44d89c115671baace7bd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:50:01.623190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:01.639919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.640977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.641274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.641550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.655063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:01.660134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:01.661096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:01.661901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 99/200] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:01.668199 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:01.749335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:01.790880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c7c2c-dcc2-42b4-8a0c-2aff4bc9623d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.830967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.831592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.832913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.832913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.832913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:01.833187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:01.834497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:01.843532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 102/201] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:01.849710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:01.959474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d526453-7906-4bd4-b4fd-4d252999faff None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:01.995520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:01.996181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:01.996461 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:01.996735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] PATH_INFO: `/v3/users/1b5e6d49a24a459ea419d8d0e29078ff` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:01.997875 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] RBAC: Authorizing `identity:get_user(user_id=1b5e6d49a24a459ea419d8d0e29078ff)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:01.999360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:02.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 100/202] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:01 2022] GET /identity/v3/users/1b5e6d49a24a459ea419d8d0e29078ff => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:02.007214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:02.118012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1b36-0f83-452c-aa37-69b539e0bda1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:02.145569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:02.155501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:02.156145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:02.156426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:02.156699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:02.226313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:02.227710 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:02.335834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 103/203] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:02.341622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:02.394628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:02.448985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97c628-c285-493d-b648-ba1babfadcdc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:02.487978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6872c-3fa2-4596-9937-e0a711d0fce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:02.488534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6872c-3fa2-4596-9937-e0a711d0fce0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:02.488811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6872c-3fa2-4596-9937-e0a711d0fce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:02.489083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6872c-3fa2-4596-9937-e0a711d0fce0 None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:02.490213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6872c-3fa2-4596-9937-e0a711d0fce0 None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:02.491734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6872c-3fa2-4596-9937-e0a711d0fce0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:02.498292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 101/204] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:02 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:02.499695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:02.530941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:02.609955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5b46-f133-48e4-ba22-2aaea85dce51 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:02.645058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a9c00-0425-4b0a-a275-705134f5e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:02.645659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a9c00-0425-4b0a-a275-705134f5e512 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:02.645934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a9c00-0425-4b0a-a275-705134f5e512 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:02.646206 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a9c00-0425-4b0a-a275-705134f5e512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:02.646919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a9c00-0425-4b0a-a275-705134f5e512 None admin] RBAC: Adding query filter params (role.id=b109a364aa914a98aee2e78986065def, scope.project.id=72927d62c8b04432867a524b73620dcb, user.id=1b5e6d49a24a459ea419d8d0e29078ff) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:02.647574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a9c00-0425-4b0a-a275-705134f5e512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:02.649050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a9c00-0425-4b0a-a275-705134f5e512 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:02.654984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 104/205] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:02 2022] GET /identity/v3/role_assignments?user.id=1b5e6d49a24a459ea419d8d0e29078ff&scope.project.id=72927d62c8b04432867a524b73620dcb&role.id=b109a364aa914a98aee2e78986065def => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:03.826526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d204220-1a6d-4adf-b44c-47607f6aa5b0 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:03.827073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d204220-1a6d-4adf-b44c-47607f6aa5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:03.827073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d204220-1a6d-4adf-b44c-47607f6aa5b0 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:03.828748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 102/206] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:03.833944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da36d85-4979-473d-9649-ae6648cba968 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:03.834324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da36d85-4979-473d-9649-ae6648cba968 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:03.834495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da36d85-4979-473d-9649-ae6648cba968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:03.853464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0da36d85-4979-473d-9649-ae6648cba968 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:03.853633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bd612-1939-47ae-bc16-cb2568d230d6 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:03.853912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bd612-1939-47ae-bc16-cb2568d230d6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:03.854189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bd612-1939-47ae-bc16-cb2568d230d6 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:03.854648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0da36d85-4979-473d-9649-ae6648cba968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:03.855670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 103/207] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:03.860525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfb63-66a8-4905-ae09-76fdd2189338 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:03.860811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfb63-66a8-4905-ae09-76fdd2189338 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:03.861089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfb63-66a8-4905-ae09-76fdd2189338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:03.879307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-72ebfb63-66a8-4905-ae09-76fdd2189338 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:03.880528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-72ebfb63-66a8-4905-ae09-76fdd2189338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:03.926197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 105/208] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:03.928218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad830a8-0529-405b-b384-8ecb98e1be6b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:03.929021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad830a8-0529-405b-b384-8ecb98e1be6b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:03.929430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad830a8-0529-405b-b384-8ecb98e1be6b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:03.930819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 106/209] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:03.937192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf6bc5-a166-4764-a3a4-420c7839d598 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:03.937473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf6bc5-a166-4764-a3a4-420c7839d598 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:03.938020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf6bc5-a166-4764-a3a4-420c7839d598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:03.948889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 104/210] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:03.957196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b7bf6bc5-a166-4764-a3a4-420c7839d598 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:03.958366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b7bf6bc5-a166-4764-a3a4-420c7839d598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:03.977128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:04.027812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 107/211] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:04.199838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfb63-66a8-4905-ae09-76fdd2189338 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.216730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf6bc5-a166-4764-a3a4-420c7839d598 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.253252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:04.312998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfb63-66a8-4905-ae09-76fdd2189338 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.324262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf6bc5-a166-4764-a3a4-420c7839d598 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.346653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:04.347295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:04.347574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:04.347885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:04.348982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:04.351017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:04.352947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 105/212] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:04.353543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.362415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:04.363040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:04.363314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:04.363581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:04.374091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:04.378865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:04.380056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 108/213] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:04.380892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.396043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:04.456012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c938a7-4963-4b3b-9f7c-67901a67745f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.470164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:04.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfad666-a529-4596-8be2-cac04b8f4991 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.490486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:04.491347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:04.491627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:04.491930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:04.501807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:04.506732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:04.507615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:04.508602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 106/214] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:04.509259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.525610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:04.526219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:04.526498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:04.526770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:04.527417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:04.528908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:04.608397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41bf12-b530-48ca-9029-0b82d8c1334c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.642245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:04.642895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:04.643177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:04.643453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:04.644140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:04.644445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:04.645827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:04.653874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 107/215] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:04.654666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.667923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 109/216] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:04 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:04.667923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.708551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:04.738906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:04.781048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bbcd6-3101-4087-879b-87008a95ef89 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.789406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206029b-b6f2-4dfe-8827-7405ead41978 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.822441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:04.823181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:04.823450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:04.823931 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] PATH_INFO: `/v3/users/3fcd0083417d485880e8362f8cf24aae` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:04.833388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:04.834832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:04.834832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:04.834832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:04.836050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:04.836360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:04.837732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:04.838608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcd0083417d485880e8362f8cf24aae)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:04.840099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:04.841782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 110/217] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/users/3fcd0083417d485880e8362f8cf24aae => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:04.847392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.854563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 108/218] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:04.860479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:04.969763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:04.973103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bc6e6-103b-42a1-a97b-03cefac48e05 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:04.973495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eadffa-5953-46f4-811a-0254c79ee37b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:05.008514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:05.021539 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:05.022238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:05.022660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:05.023086 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] PATH_INFO: `/v3/users/126d9e96a4194354828f2e39799c3427` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:05.024348 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] RBAC: Authorizing `identity:get_user(user_id=126d9e96a4194354828f2e39799c3427)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:05.025547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:05.026018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:05.026121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:05.026405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:05.026674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:05.027400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] RBAC: Adding query filter params (role.id=b109a364aa914a98aee2e78986065def, user.id=3fcd0083417d485880e8362f8cf24aae, scope.system=all) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:05.027707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:05.028041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 109/219] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/users/126d9e96a4194354828f2e39799c3427 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:05.033235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:05.038747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:05.043289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 111/220] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 26 10:50:04 2022] GET /identity/v3/role_assignments?user.id=3fcd0083417d485880e8362f8cf24aae&scope.system=all&role.id=b109a364aa914a98aee2e78986065def => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:05.165719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d70a13-969b-445e-9687-61640e55bf4e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:05.205656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:05.206289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:05.206633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:05.206915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:05.248169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:05.288943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:05.290485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:05.400202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 110/221] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:05.405604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:05.509562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e49fb5-393c-4a89-bd20-981333985f5f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:05.550082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351e338-f68c-4b78-be9c-1c18b6dc8fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:05.550730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351e338-f68c-4b78-be9c-1c18b6dc8fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:05.551009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351e338-f68c-4b78-be9c-1c18b6dc8fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:05.551284 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351e338-f68c-4b78-be9c-1c18b6dc8fa5 None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:05.552509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351e338-f68c-4b78-be9c-1c18b6dc8fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:05.554405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351e338-f68c-4b78-be9c-1c18b6dc8fa5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:05.556268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 112/222] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:05 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:05.561993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:05.634645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:05.670503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b51893-9483-47d6-ae4c-d038fa1c01f6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:05.706629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f3f62-0c3c-495c-abe6-75d6a50209c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:05.707241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f3f62-0c3c-495c-abe6-75d6a50209c2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:05.707520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f3f62-0c3c-495c-abe6-75d6a50209c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:05.707817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f3f62-0c3c-495c-abe6-75d6a50209c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:05.708549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f3f62-0c3c-495c-abe6-75d6a50209c2 None admin] RBAC: Adding query filter params (role.id=60ffc030f91c44d89c115671baace7bd, scope.project.id=72927d62c8b04432867a524b73620dcb, user.id=126d9e96a4194354828f2e39799c3427) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:05.708856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f3f62-0c3c-495c-abe6-75d6a50209c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:05.710359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f3f62-0c3c-495c-abe6-75d6a50209c2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:05.716961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 111/223] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:05 2022] GET /identity/v3/role_assignments?user.id=126d9e96a4194354828f2e39799c3427&scope.project.id=72927d62c8b04432867a524b73620dcb&role.id=60ffc030f91c44d89c115671baace7bd => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:07.023584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b8d1-e0b0-4e8a-b706-1bb146f20a53 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.024143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b8d1-e0b0-4e8a-b706-1bb146f20a53 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.024143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b8d1-e0b0-4e8a-b706-1bb146f20a53 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.025646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 113/224] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:07.030752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea725e-adb2-4dfe-a880-958e2cdd78b6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.031125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea725e-adb2-4dfe-a880-958e2cdd78b6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.031315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea725e-adb2-4dfe-a880-958e2cdd78b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.051265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-06ea725e-adb2-4dfe-a880-958e2cdd78b6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:07.052591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-06ea725e-adb2-4dfe-a880-958e2cdd78b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:07.080794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e01b-6ca0-43f5-b7a9-5846b36a6f8b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.081077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e01b-6ca0-43f5-b7a9-5846b36a6f8b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.081515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e01b-6ca0-43f5-b7a9-5846b36a6f8b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.083009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 114/225] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:07.088620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ec6e5-42f5-44f4-bd3e-c3b48a5ac3e5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.088908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ec6e5-42f5-44f4-bd3e-c3b48a5ac3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.089185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ec6e5-42f5-44f4-bd3e-c3b48a5ac3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.109805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-826ec6e5-42f5-44f4-bd3e-c3b48a5ac3e5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:07.111016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-826ec6e5-42f5-44f4-bd3e-c3b48a5ac3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:07.123969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 112/226] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:07.168870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:07.184437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 115/227] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:07.289368 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368ee36-da91-4a38-bfcb-9f89b4f6701d None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.289652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368ee36-da91-4a38-bfcb-9f89b4f6701d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.289836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368ee36-da91-4a38-bfcb-9f89b4f6701d None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.291326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 113/228] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:07.296238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a66c8-9ecf-4b07-aa02-271fd525c22f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.296524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a66c8-9ecf-4b07-aa02-271fd525c22f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.296796 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a66c8-9ecf-4b07-aa02-271fd525c22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.314869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-cc9a66c8-9ecf-4b07-aa02-271fd525c22f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:07.316058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-cc9a66c8-9ecf-4b07-aa02-271fd525c22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:07.384046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 116/229] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:07.396926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368ee36-da91-4a38-bfcb-9f89b4f6701d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:07.456986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a66c8-9ecf-4b07-aa02-271fd525c22f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:07.460808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:07.504289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368ee36-da91-4a38-bfcb-9f89b4f6701d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:07.542503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:07.543066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.543343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.543614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.557477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:07.562447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:07.563501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:07.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:07.567170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a66c8-9ecf-4b07-aa02-271fd525c22f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:07.568327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 114/230] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:07.569198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:07.588959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:07.608137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:07.608984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.609410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.609868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.610976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:07.612651 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:07.614233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 117/231] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:07.620878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:07.673734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55624654-35f3-4d2e-885d-316bb29de666 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:07.693556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:07.707836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:07.708444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.708723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.708995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.710244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:07.710550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:07.711986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:07.719876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 115/232] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:07.720750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:07.726588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e584-4e62-4a64-9498-174086aee790 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:07.762125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:07.762733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.763013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.763285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] PATH_INFO: `/v3/groups` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.763956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:07.765385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:07.820819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9a57-54bf-4c33-8118-1e3ba658dd0b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:07.843458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 118/233] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Oct 26 10:50:07 2022] POST /identity/v3/groups => generated 229 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:50:07.843458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:07.859984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:07.860743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:07.861032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:07.861311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:07.874090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:07.879726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:07.880715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:07.882046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 116/234] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:07.888466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:07.908619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:07.937645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:07.973478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ee17b-a140-4875-9cbb-5f992dd9c742 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:08.000390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b099d-991f-4344-81ae-a6fd25fb8b56 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:08.011309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:08.012039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:08.012859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:08.013141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] PATH_INFO: `/v3/users/1dd167d1935544a5a7685f8e9b310e46` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:08.028518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] RBAC: Authorizing `identity:get_user(user_id=1dd167d1935544a5a7685f8e9b310e46)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:08.030557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:08.032518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 119/235] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:07 2022] GET /identity/v3/users/1dd167d1935544a5a7685f8e9b310e46 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:08.039320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:08.050912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:08.054358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:08.054358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:08.054358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:08.054358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:08.054358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:08.055886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:08.065878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 117/236] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:08.072804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:08.144732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196253f3-a7f3-415f-8e65-cee7839d7015 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:08.147301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:08.181679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:08.182063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30358ca2-b1f9-47c4-a821-71237bf3541e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:08.182295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:08.182575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:08.182850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:08.183575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] RBAC: Adding query filter params (role.id=60ffc030f91c44d89c115671baace7bd, user.id=1dd167d1935544a5a7685f8e9b310e46, scope.system=all) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:08.183917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:08.185284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:08.185389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:08.190823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 120/237] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 26 10:50:08 2022] GET /identity/v3/role_assignments?user.id=1dd167d1935544a5a7685f8e9b310e46&scope.system=all&role.id=60ffc030f91c44d89c115671baace7bd => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:08.219994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e56e2-3d20-4a36-bfb8-50dce48287f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:08.220657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e56e2-3d20-4a36-bfb8-50dce48287f0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:08.220935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e56e2-3d20-4a36-bfb8-50dce48287f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:08.221209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e56e2-3d20-4a36-bfb8-50dce48287f0 None admin] PATH_INFO: `/v3/users/3fcd0083417d485880e8362f8cf24aae` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:08.222560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e56e2-3d20-4a36-bfb8-50dce48287f0 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcd0083417d485880e8362f8cf24aae)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:08.224081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e56e2-3d20-4a36-bfb8-50dce48287f0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:08.226048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 118/238] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:08 2022] GET /identity/v3/users/3fcd0083417d485880e8362f8cf24aae => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:08.231970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:08.343869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee19472-458f-4488-9384-67a1448d4af1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:08.380903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c48ddc-3fc6-476e-9688-7f895f477e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:08.381835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48ddc-3fc6-476e-9688-7f895f477e0e None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:08.382126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48ddc-3fc6-476e-9688-7f895f477e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:08.382401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48ddc-3fc6-476e-9688-7f895f477e0e None admin] PATH_INFO: `/v3/system/users/3fcd0083417d485880e8362f8cf24aae/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:08.383949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c48ddc-3fc6-476e-9688-7f895f477e0e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3fcd0083417d485880e8362f8cf24aae, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:08.385472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c48ddc-3fc6-476e-9688-7f895f477e0e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:08.479038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 121/239] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Oct 26 10:50:08 2022] PUT /identity/v3/system/users/3fcd0083417d485880e8362f8cf24aae/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:50:10.048365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3625a8-d708-4dc5-b6d6-744a203a489b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.048825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3625a8-d708-4dc5-b6d6-744a203a489b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.048899 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3625a8-d708-4dc5-b6d6-744a203a489b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.050416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 119/240] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:10.056174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db99c85-4d36-4a4e-bb19-3f798f02c9e1 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.056627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db99c85-4d36-4a4e-bb19-3f798f02c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.056806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db99c85-4d36-4a4e-bb19-3f798f02c9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.075225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3db99c85-4d36-4a4e-bb19-3f798f02c9e1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:10.076446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3db99c85-4d36-4a4e-bb19-3f798f02c9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:10.104338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:10.147685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 122/241] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:10.344721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c0ce-6366-450f-912e-1690ef0e53bd None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.345035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c0ce-6366-450f-912e-1690ef0e53bd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.345216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c0ce-6366-450f-912e-1690ef0e53bd None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.346679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 120/242] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:10.351979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b51db-2bf0-4194-8890-136a5e97732b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.352271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b51db-2bf0-4194-8890-136a5e97732b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.352542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b51db-2bf0-4194-8890-136a5e97732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.370875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-522b51db-2bf0-4194-8890-136a5e97732b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:10.372080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-522b51db-2bf0-4194-8890-136a5e97732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:10.424953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23c0ce-6366-450f-912e-1690ef0e53bd None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:10.439496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 123/243] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:10.534338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23c0ce-6366-450f-912e-1690ef0e53bd None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:10.569274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3e51d-9678-481c-b853-09c40bc6c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:10.569817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3e51d-9678-481c-b853-09c40bc6c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.570096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3e51d-9678-481c-b853-09c40bc6c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.570367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3e51d-9678-481c-b853-09c40bc6c4d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.571488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3e51d-9678-481c-b853-09c40bc6c4d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:10.573043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3e51d-9678-481c-b853-09c40bc6c4d3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:10.574768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 121/244] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:10.580134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b51db-2bf0-4194-8890-136a5e97732b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:10.606061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabae6-8461-4f7b-ab59-cae0be2b35f8 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.606345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabae6-8461-4f7b-ab59-cae0be2b35f8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.606618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabae6-8461-4f7b-ab59-cae0be2b35f8 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.607984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 122/245] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:10.612833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99d1d-8545-49c1-b7e0-493f7925d832 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.613108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99d1d-8545-49c1-b7e0-493f7925d832 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.613372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99d1d-8545-49c1-b7e0-493f7925d832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.632550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a8d99d1d-8545-49c1-b7e0-493f7925d832 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:10.633709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a8d99d1d-8545-49c1-b7e0-493f7925d832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:10.669271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:10.684635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:10.693254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b51db-2bf0-4194-8890-136a5e97732b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:10.704640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 123/246] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:10.710008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d99d1d-8545-49c1-b7e0-493f7925d832 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:10.734150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:10.734768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.735038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.735304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] PATH_INFO: `/v3/groups` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.736039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:10.737458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:10.810013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d99d1d-8545-49c1-b7e0-493f7925d832 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:10.841804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 124/247] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Oct 26 10:50:10 2022] POST /identity/v3/groups => generated 226 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:10.852305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:10.852962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:10.853243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:10.853512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:10.866415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:10.871556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:10.872549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:10.876699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 124/248] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:10.879273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:10.971622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:10.998672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331fa4-9298-4fc2-968e-1ba86172fc64 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:11.015714 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:11.037176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:11.038813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:11.038813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:11.039104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:11.039707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:11.040090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:11.041494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:11.051189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 125/249] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:11.057240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:11.077159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:11.105585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d410f6d-6a03-4cb2-acc1-c0908b447934 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:11.152251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:11.152870 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:11.153147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:11.153468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:11.170930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:11.176387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:11.177273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:11.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 125/250] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:11.185004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:11.195000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea74c-229a-4f1a-962e-8aaf84c3a46e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:11.235844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:11.237101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:11.237621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:11.237897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] PATH_INFO: `/v3/users/1dd167d1935544a5a7685f8e9b310e46` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:11.240693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] RBAC: Authorizing `identity:get_user(user_id=1dd167d1935544a5a7685f8e9b310e46)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:11.242407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:11.246581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 126/251] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:11 2022] GET /identity/v3/users/1dd167d1935544a5a7685f8e9b310e46 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:11.251823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:11.315183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:11.328936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637801e6-2950-4d92-8890-b267630a5edc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:11.372954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:11.373570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:11.373847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:11.374116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:11.374902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:11.375203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:11.377813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:11.387455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 126/252] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:11.393806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:11.399772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:11.423273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f9ad-5b18-470c-89c6-dffebfb1924b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:11.471197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:11.472226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:11.472642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:11.473049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] PATH_INFO: `/v3/system/users/1dd167d1935544a5a7685f8e9b310e46/roles/60ffc030f91c44d89c115671baace7bd` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:11.475512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1dd167d1935544a5a7685f8e9b310e46, role_id=60ffc030f91c44d89c115671baace7bd)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:11.477692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:11.528002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf488-06dc-4f55-9046-8be7745df92b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:11.551814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 127/253] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Oct 26 10:50:11 2022] PUT /identity/v3/system/users/1dd167d1935544a5a7685f8e9b310e46/roles/60ffc030f91c44d89c115671baace7bd => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:11.574071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcefbf0-fd62-4e90-8248-e2bb89f2a945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:11.575130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcefbf0-fd62-4e90-8248-e2bb89f2a945 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:11.575637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcefbf0-fd62-4e90-8248-e2bb89f2a945 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:11.576133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcefbf0-fd62-4e90-8248-e2bb89f2a945 None admin] PATH_INFO: `/v3/users/3fcd0083417d485880e8362f8cf24aae` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:11.577616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcefbf0-fd62-4e90-8248-e2bb89f2a945 None admin] RBAC: Authorizing `identity:get_user(user_id=3fcd0083417d485880e8362f8cf24aae)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:11.579400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcefbf0-fd62-4e90-8248-e2bb89f2a945 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:11.586934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 127/254] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:11 2022] GET /identity/v3/users/3fcd0083417d485880e8362f8cf24aae => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:11.588803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:11.721280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8199c74-07e4-4941-8d05-2769d8454964 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:11.767372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049949a-5508-4cb4-b6e3-42af96af8e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:11.768023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049949a-5508-4cb4-b6e3-42af96af8e51 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:11.768308 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049949a-5508-4cb4-b6e3-42af96af8e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:11.768582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049949a-5508-4cb4-b6e3-42af96af8e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:11.769315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049949a-5508-4cb4-b6e3-42af96af8e51 None admin] RBAC: Adding query filter params (role.id=b109a364aa914a98aee2e78986065def, user.id=3fcd0083417d485880e8362f8cf24aae, scope.system=all) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:11.769631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049949a-5508-4cb4-b6e3-42af96af8e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:11.771205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049949a-5508-4cb4-b6e3-42af96af8e51 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:11.785226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 128/255] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 26 10:50:11 2022] GET /identity/v3/role_assignments?user.id=3fcd0083417d485880e8362f8cf24aae&scope.system=all&role.id=b109a364aa914a98aee2e78986065def => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:13.299886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d2a1-359a-44d8-91f9-de502aa92572 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:13.300353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d2a1-359a-44d8-91f9-de502aa92572 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:13.300353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d2a1-359a-44d8-91f9-de502aa92572 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:13.301935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 128/256] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:13.308407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4260734-0033-4bcb-ac64-1519a11804fe None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:13.308820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4260734-0033-4bcb-ac64-1519a11804fe None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:13.308994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4260734-0033-4bcb-ac64-1519a11804fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:13.318584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:13.329802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a4260734-0033-4bcb-ac64-1519a11804fe None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:13.339726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a4260734-0033-4bcb-ac64-1519a11804fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:13.412014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 129/257] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 10:50:13.603438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecefe3-dc71-46b3-b42b-b26b63a2b0e7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:13.603762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecefe3-dc71-46b3-b42b-b26b63a2b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:13.604012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecefe3-dc71-46b3-b42b-b26b63a2b0e7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:13.605618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 129/258] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:13.610900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe3892-a250-4911-8f4a-938065db7ce5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:13.611187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe3892-a250-4911-8f4a-938065db7ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:13.611464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe3892-a250-4911-8f4a-938065db7ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:13.640676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e3fe3892-a250-4911-8f4a-938065db7ce5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:13.641883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e3fe3892-a250-4911-8f4a-938065db7ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:13.650675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 130/259] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 10:50:14.013471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecefe3-dc71-46b3-b42b-b26b63a2b0e7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.028457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecefe3-dc71-46b3-b42b-b26b63a2b0e7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.036406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.037049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.037383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.038081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.053293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:14.058473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:14.059323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:14.060161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:14.062202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 130/260] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:14.067129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe3892-a250-4911-8f4a-938065db7ce5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.081204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe3892-a250-4911-8f4a-938065db7ce5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.096979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.097606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.097891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.098168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.098910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:14.099219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.101136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 10:50:14.101136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 10:50:14.101136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 10:50:14.102070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.114537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 131/261] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:14.121439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.137429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80f131-cbc3-4014-95f4-e4316fc8192b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.146360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.147548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.149664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.149664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.162346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:14.166233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.168089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 10:50:14.168089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 10:50:14.168089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 10:50:14.169331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.171130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 131/262] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:14.176889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.190009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dbedd-f802-4ec1-b676-b1e27d9e04a9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.205040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.205699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.205978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.206251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.207415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.210314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.211941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 132/263] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:14.217815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.232691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f66f3-8a6e-4a48-9b28-eb4089faaef5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.243577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.244287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.244567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.244844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.253125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa6e65e21f64df995b975bfb4ebdbba, group_id=06f307e7c0124171b92ab5dfc2d21f8d, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.254166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 10:50:14.254166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 10:50:14.254166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 10:50:14.256295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.268930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.287829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0087cfd-b1a5-4630-a533-eb5a3d5a9e17 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.296970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.300463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.300463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.300463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.314654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:14.320239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:14.322278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:14.324227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 133/264] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:14.330684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.337849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6fcef-6d6c-4ecc-9580-d13bf82d2bc8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.347907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.348517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.348833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.349110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.349936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:14.350244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.351896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.367771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 134/265] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:14.373264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.383739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd41285-338d-4ec9-a1c8-40fcef8dcc92 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.392007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.392654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.392942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.393223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.400496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.406106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 10:50:14.406106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 10:50:14.406106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 10:50:14.406106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.406106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 135/266] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:14.410113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.412622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 132/267] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 26 10:50:14 2022] PUT /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:14.420657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ff2b7-d650-4a14-956e-7dfd4f04c4b0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.496938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b408245-5c31-49b9-a52f-a14d82a8d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.497667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408245-5c31-49b9-a52f-a14d82a8d2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.497831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408245-5c31-49b9-a52f-a14d82a8d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.498113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408245-5c31-49b9-a52f-a14d82a8d2b7 admin admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.499245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b408245-5c31-49b9-a52f-a14d82a8d2b7 admin admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.501363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b408245-5c31-49b9-a52f-a14d82a8d2b7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.503296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 136/268] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 26 10:50:14 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:14.509913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:14.534672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3b3a6-2694-4b8a-95a2-e640f7462699 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:14.546031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093a0b8-3a4d-4b29-bc51-538029455955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:14.546725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093a0b8-3a4d-4b29-bc51-538029455955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.547005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093a0b8-3a4d-4b29-bc51-538029455955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.547279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093a0b8-3a4d-4b29-bc51-538029455955 admin admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/a2f657dd4fab43288cb9e84eb6d94d51` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.556687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093a0b8-3a4d-4b29-bc51-538029455955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa6e65e21f64df995b975bfb4ebdbba, group_id=06f307e7c0124171b92ab5dfc2d21f8d, role_id=a2f657dd4fab43288cb9e84eb6d94d51)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:14.559486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093a0b8-3a4d-4b29-bc51-538029455955 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:14.702433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 133/269] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 26 10:50:14 2022] PUT /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/a2f657dd4fab43288cb9e84eb6d94d51 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:14.858285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7818a-43b7-4bcb-84d6-2a1f2ff55e17 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.858674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7818a-43b7-4bcb-84d6-2a1f2ff55e17 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.858857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7818a-43b7-4bcb-84d6-2a1f2ff55e17 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.860548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 137/270] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:14.867193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad99539-b1a2-4168-89d6-36fa1c84522c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.867480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad99539-b1a2-4168-89d6-36fa1c84522c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.867763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad99539-b1a2-4168-89d6-36fa1c84522c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.883834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae544b1-ae34-4d7e-8e76-b75a057f40dc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.883834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae544b1-ae34-4d7e-8e76-b75a057f40dc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.883834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae544b1-ae34-4d7e-8e76-b75a057f40dc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.883834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 138/271] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:14.891800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1974-0af2-4a8f-a1f7-ced314a608cb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:14.891800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1974-0af2-4a8f-a1f7-ced314a608cb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:14.891800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe1974-0af2-4a8f-a1f7-ced314a608cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:14.899972 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cad99539-b1a2-4168-89d6-36fa1c84522c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:14.901257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cad99539-b1a2-4168-89d6-36fa1c84522c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:14.914846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6ffe1974-0af2-4a8f-a1f7-ced314a608cb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:14.916289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6ffe1974-0af2-4a8f-a1f7-ced314a608cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:14.984010 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:14.997309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 139/272] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 10:50:15.025618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 134/273] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:15.301874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1974-0af2-4a8f-a1f7-ced314a608cb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:15.311548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe1974-0af2-4a8f-a1f7-ced314a608cb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:15.316236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:15.316940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:15.317388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:15.317810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:15.335911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:15.341163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:15.342215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:15.343227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:15.345312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 140/274] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 26 10:50:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:15.352461 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad99539-b1a2-4168-89d6-36fa1c84522c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:15.365405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad99539-b1a2-4168-89d6-36fa1c84522c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:15.375528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:15.376177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:15.376453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:15.376724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:15.377400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:15.377700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:15.378720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 10:50:15.378720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 10:50:15.378720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 10:50:15.379631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:15.395477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 135/275] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:50:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:15.401340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:15.418045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f0747-179b-42b4-a9cb-1fb924740a39 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:15.429667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:15.430612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:15.431069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:15.431512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:15.432852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:15.435028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:15.436845 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 141/276] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 26 10:50:15 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:15.442472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:15.442611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:15.454450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754c03c-1c34-467b-ac8e-12e98d9dec9d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:15.459478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:15.460209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:15.460489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:15.460762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:15.461909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:15.464021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:15.465971 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 142/277] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 26 10:50:15 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:15.471568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:15.482538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19535d1-5ebb-42af-92de-0fe289ab8d89 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:15.487271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:15.487959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:15.488239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:15.488515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:15.490732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72927d62c8b04432867a524b73620dcb, group_id=06f307e7c0124171b92ab5dfc2d21f8d, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:15.492236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 10:50:15.492236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 10:50:15.492236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 10:50:15.494358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:15.604422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 143/278] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 26 10:50:15 2022] PUT /identity/v3/projects/72927d62c8b04432867a524b73620dcb/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:50:15.605486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:15.611194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2a2ae-b6f9-4572-81da-adf72d2bd86c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:15.662716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:15.663365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:15.663646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:15.664060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:15.680083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:15.685365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:15.686268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:15.687137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 136/279] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:15.692920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:15.843238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a000-28b4-4da5-9260-02704277bb32 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:15.888731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe350b-7262-495b-86cc-ccaaf6b29bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:15.889743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe350b-7262-495b-86cc-ccaaf6b29bfe None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:15.890230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe350b-7262-495b-86cc-ccaaf6b29bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:15.890707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe350b-7262-495b-86cc-ccaaf6b29bfe None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:15.891606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe350b-7262-495b-86cc-ccaaf6b29bfe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:15.892167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe350b-7262-495b-86cc-ccaaf6b29bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:15.893861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe350b-7262-495b-86cc-ccaaf6b29bfe None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:15.908464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 144/280] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:15.914904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.070446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504eea7-8491-46d8-ba68-3c417d5099f6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.076934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5e3bb-de19-4650-8177-63dbe5480bab None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.077341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5e3bb-de19-4650-8177-63dbe5480bab None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.077723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5e3bb-de19-4650-8177-63dbe5480bab None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.079550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 145/281] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:16.084750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373e15-5f92-46da-b424-9249a729cdf0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.085026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373e15-5f92-46da-b424-9249a729cdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.085442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373e15-5f92-46da-b424-9249a729cdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.123904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.124601 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.124887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.125161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] PATH_INFO: `/v3/users/1dd167d1935544a5a7685f8e9b310e46` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.125261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a4373e15-5f92-46da-b424-9249a729cdf0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:16.126343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] RBAC: Authorizing `identity:get_user(user_id=1dd167d1935544a5a7685f8e9b310e46)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:16.126603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a4373e15-5f92-46da-b424-9249a729cdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:16.127825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:16.129723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 137/282] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:50:15 2022] GET /identity/v3/users/1dd167d1935544a5a7685f8e9b310e46 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:16.137018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.171470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:16.180934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:16.261558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275f91b-1bba-4e66-aa08-d2eda11fa3ec None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.301405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 146/283] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 10:50:16.303477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a3c2-2811-4e78-bc74-2a84e304d5fc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.304045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.304139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a3c2-2811-4e78-bc74-2a84e304d5fc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.304139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a3c2-2811-4e78-bc74-2a84e304d5fc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.304410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.304644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.304920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.305305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 147/284] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:16.305654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] RBAC: Adding query filter params (role.id=60ffc030f91c44d89c115671baace7bd, user.id=1dd167d1935544a5a7685f8e9b310e46, scope.system=all) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:16.305964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:16.307472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:16.313212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab72f3-6920-43c3-aa0a-50b9f088bf3e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.313492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab72f3-6920-43c3-aa0a-50b9f088bf3e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.313765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab72f3-6920-43c3-aa0a-50b9f088bf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.327625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 138/285] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/role_assignments?user.id=1dd167d1935544a5a7685f8e9b310e46&scope.system=all&role.id=60ffc030f91c44d89c115671baace7bd => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:16.350344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1dab72f3-6920-43c3-aa0a-50b9f088bf3e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:16.351295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1dab72f3-6920-43c3-aa0a-50b9f088bf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:16.363764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 148/286] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 26 10:50:16.727308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.739485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dfc3d-5df9-4d68-952d-c5064d0312b0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.744386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.745023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.745307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.745577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.758312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:16.764018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:16.764883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:16.765673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 139/287] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:16.771029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab72f3-6920-43c3-aa0a-50b9f088bf3e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.780581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab72f3-6920-43c3-aa0a-50b9f088bf3e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.787558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.788465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.788465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.788786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.789397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:16.789697 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:16.791300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:16.800643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 149/288] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:16.807070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.819245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f55a5b-7312-4e78-b68d-7b1116df3ed4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.823546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.824204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.824485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.824757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.825924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:16.827879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:16.829630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 140/289] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:16.836401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.846713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61341df7-bb65-4d20-8227-aa7d95748b0f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.852362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.853046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.853372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.853663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.854896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:16.857073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:16.860540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 150/290] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:16.865461 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.875435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bafb7-177c-4785-a678-3862e53309fe admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.880352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.881576 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.882170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.882170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/a2f657dd4fab43288cb9e84eb6d94d51` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.884409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72927d62c8b04432867a524b73620dcb, group_id=06f307e7c0124171b92ab5dfc2d21f8d, role_id=a2f657dd4fab43288cb9e84eb6d94d51)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:16.887512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:16.898716 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.909763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb441e-368e-42de-b095-0c2932ec8c55 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.915563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.916249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.916556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.916883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.929216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:16.935029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:16.945511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 151/291] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:16.947385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:16.953812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f4fdb-4bb6-4527-adc8-fbedfcc65f1e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:16.958610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:16.960133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:16.960403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:16.960555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:16.961219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:16.961566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:16.966263 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:16.970472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 141/292] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 26 10:50:16 2022] PUT /identity/v3/projects/72927d62c8b04432867a524b73620dcb/groups/06f307e7c0124171b92ab5dfc2d21f8d/roles/a2f657dd4fab43288cb9e84eb6d94d51 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:16.985017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 152/293] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:16.990616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:17.109385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356861fa-3a68-4a5f-98ba-e87f1dd927b6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:17.118480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:17.119143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:17.119423 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:17.119698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] PATH_INFO: `/v3/groups/7e3971becff64204b5dc4c3630c85271` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:17.121959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:17.126474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] RBAC: Authorizing `identity:get_group(group_id=7e3971becff64204b5dc4c3630c85271)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:17.128457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:17.130399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 142/294] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 26 10:50:16 2022] GET /identity/v3/groups/7e3971becff64204b5dc4c3630c85271 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:17.136331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:17.154362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4718e-e751-43ef-a415-2df0512c3000 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:17.164655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8cfe-83f9-46bc-afad-a20afcde31ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:17.165406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8cfe-83f9-46bc-afad-a20afcde31ff admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:17.165744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8cfe-83f9-46bc-afad-a20afcde31ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:17.166076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8cfe-83f9-46bc-afad-a20afcde31ff admin admin] PATH_INFO: `/v3/projects/b00b7a3900004d669310a34e206f2fc2` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:17.169658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d8cfe-83f9-46bc-afad-a20afcde31ff admin admin] RBAC: Authorizing `identity:get_project(project_id=b00b7a3900004d669310a34e206f2fc2)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:17.171609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d8cfe-83f9-46bc-afad-a20afcde31ff admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:17.173763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 153/295] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 26 10:50:17 2022] GET /identity/v3/projects/b00b7a3900004d669310a34e206f2fc2 => generated 344 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:17.179574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:17.193649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf91ae-407f-4206-84e0-3a5f67482aa6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:17.203811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b3766-854e-495c-b57b-c0bb29acb3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:17.204504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b3766-854e-495c-b57b-c0bb29acb3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:17.204783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b3766-854e-495c-b57b-c0bb29acb3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:17.205052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b3766-854e-495c-b57b-c0bb29acb3bd admin admin] PATH_INFO: `/v3/projects/b00b7a3900004d669310a34e206f2fc2/groups/7e3971becff64204b5dc4c3630c85271/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:17.214442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b3766-854e-495c-b57b-c0bb29acb3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00b7a3900004d669310a34e206f2fc2, group_id=7e3971becff64204b5dc4c3630c85271, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:17.217155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b3766-854e-495c-b57b-c0bb29acb3bd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:17.294897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 143/296] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 26 10:50:17 2022] PUT /identity/v3/projects/b00b7a3900004d669310a34e206f2fc2/groups/7e3971becff64204b5dc4c3630c85271/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:17.937537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda411b1-3ab4-458a-a654-f98c2ae4278b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:17.938146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda411b1-3ab4-458a-a654-f98c2ae4278b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:17.938146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda411b1-3ab4-458a-a654-f98c2ae4278b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:17.939663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 154/297] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:17.945605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cbbc1-9dc5-437d-bdb1-4c998e6c88c7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:17.946061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cbbc1-9dc5-437d-bdb1-4c998e6c88c7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:17.946141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cbbc1-9dc5-437d-bdb1-4c998e6c88c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:17.965944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7c4cbbc1-9dc5-437d-bdb1-4c998e6c88c7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:17.967137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7c4cbbc1-9dc5-437d-bdb1-4c998e6c88c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:17.987647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb47f1-7707-4b1e-b810-60f87879b189 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:17.987989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb47f1-7707-4b1e-b810-60f87879b189 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:17.988211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb47f1-7707-4b1e-b810-60f87879b189 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:17.989686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 155/298] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:17.996031 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103d905-6d2a-416c-afd4-79098c31ee0b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:17.996597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103d905-6d2a-416c-afd4-79098c31ee0b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:17.997183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103d905-6d2a-416c-afd4-79098c31ee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:18.029648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2103d905-6d2a-416c-afd4-79098c31ee0b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:18.031205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2103d905-6d2a-416c-afd4-79098c31ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:18.057322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 144/299] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:18.153922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 156/300] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:18.524870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103d905-6d2a-416c-afd4-79098c31ee0b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:18.525930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cbbc1-9dc5-437d-bdb1-4c998e6c88c7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:18.612035 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:18.658200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103d905-6d2a-416c-afd4-79098c31ee0b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:18.701588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:18.702240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:18.702522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:18.702799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:18.720609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cbbc1-9dc5-437d-bdb1-4c998e6c88c7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:18.727663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:18.732603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:18.733522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:18.734352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 157/301] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:18.751389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:18.769289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:18.801556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:18.802170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:18.802452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:18.803258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:18.830973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:18.837262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:18.839510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:18.841445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 145/302] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:50:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:18.851013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:18.968865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c422520-8957-4a38-9230-9d67b22a2582 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:18.988925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad923b4-8673-484a-82ee-5857255a0bbd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:18.989846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:19.014078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ab1a0-87a9-4946-bad6-ff8c4b748689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:19.014818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab1a0-87a9-4946-bad6-ff8c4b748689 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.015126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab1a0-87a9-4946-bad6-ff8c4b748689 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.015396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab1a0-87a9-4946-bad6-ff8c4b748689 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.016104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ab1a0-87a9-4946-bad6-ff8c4b748689 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:19.016411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ab1a0-87a9-4946-bad6-ff8c4b748689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:19.017783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ab1a0-87a9-4946-bad6-ff8c4b748689 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:19.030170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 158/303] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:19.034248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57dff-66e0-4534-a4af-6713bd2e0c00 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.034248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57dff-66e0-4534-a4af-6713bd2e0c00 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.034248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57dff-66e0-4534-a4af-6713bd2e0c00 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.034248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 159/304] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:19.036692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384210f-2257-4afc-b9d5-f111a8a7acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:19.039817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384210f-2257-4afc-b9d5-f111a8a7acca None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.039817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384210f-2257-4afc-b9d5-f111a8a7acca None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.039817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384210f-2257-4afc-b9d5-f111a8a7acca None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.039817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384210f-2257-4afc-b9d5-f111a8a7acca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:19.039817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384210f-2257-4afc-b9d5-f111a8a7acca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:19.043307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384210f-2257-4afc-b9d5-f111a8a7acca None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:19.043443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57dff-66e0-4534-a4af-6713bd2e0c00 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:19.061590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 146/305] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:19.062618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0668c-199d-4df7-b929-77a1e859a143 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.062907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0668c-199d-4df7-b929-77a1e859a143 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.063184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0668c-199d-4df7-b929-77a1e859a143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.090449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a3c0668c-199d-4df7-b929-77a1e859a143 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:19.092132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a3c0668c-199d-4df7-b929-77a1e859a143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:19.133641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:19.167043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57dff-66e0-4534-a4af-6713bd2e0c00 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:19.184787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 147/306] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:19.185653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0668c-199d-4df7-b929-77a1e859a143 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:19.232588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:19.233215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.234881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.234881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.308858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:19.331441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:19.332865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:19.392004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0668c-199d-4df7-b929-77a1e859a143 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:19.463442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:19.464090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.464368 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.464644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.481189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 160/307] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:19.485707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:19.550252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:19.587489 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:19.588916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:19.604099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:19.635251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c846-ff27-48aa-a6eb-7c3b3dba5880 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:19.689118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:19.689761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.690039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.690315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.692868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:19.696080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:19.698966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 161/308] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:19 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:19.699996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:19.720789 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 148/309] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:19.722923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:19.810396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:19.830980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4004b-019e-47a0-be68-5f38364882ff None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:19.850965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc971460-7b24-480a-8e15-8eede4b4b742 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:19.884367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ebc09-38c0-489d-8072-c9101d4dacca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:19.885017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebc09-38c0-489d-8072-c9101d4dacca None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.885292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebc09-38c0-489d-8072-c9101d4dacca None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.885566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebc09-38c0-489d-8072-c9101d4dacca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.893598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:19.894236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.894519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.894792 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.895930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:19.897473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ebc09-38c0-489d-8072-c9101d4dacca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:19.898010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:19.903934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 149/310] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 26 10:50:19 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:19.903934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-023ebc09-38c0-489d-8072-c9101d4dacca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:19.904980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:19.905839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:19.906625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 162/311] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:19.912964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558c3ae-5a39-4f58-8d6b-d34873423687 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:19.912964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558c3ae-5a39-4f58-8d6b-d34873423687 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:19.912964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558c3ae-5a39-4f58-8d6b-d34873423687 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:19.914300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 163/312] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:19.916695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558c3ae-5a39-4f58-8d6b-d34873423687 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:19.930050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:20.038493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7043e7-61ca-4610-ad49-06258741232a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.060968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558c3ae-5a39-4f58-8d6b-d34873423687 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.094725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.095393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.095673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.096467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] PATH_INFO: `/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.098809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] RBAC: Authorizing `identity:get_project(project_id=3fa6e65e21f64df995b975bfb4ebdbba)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:20.100613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:20.103097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 150/313] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:19 2022] GET /identity/v3/projects/3fa6e65e21f64df995b975bfb4ebdbba => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:20.103283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.109580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:20.112513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942aa16-6ddf-4981-a40a-480a3b24b1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.113328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942aa16-6ddf-4981-a40a-480a3b24b1cd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.113798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942aa16-6ddf-4981-a40a-480a3b24b1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.114231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942aa16-6ddf-4981-a40a-480a3b24b1cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.115494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942aa16-6ddf-4981-a40a-480a3b24b1cd None admin] RBAC: Adding query filter params (group.id=06f307e7c0124171b92ab5dfc2d21f8d, role.id=b109a364aa914a98aee2e78986065def, scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:20.115819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942aa16-6ddf-4981-a40a-480a3b24b1cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:20.117298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942aa16-6ddf-4981-a40a-480a3b24b1cd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:20.124488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 164/314] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 26 10:50:19 2022] GET /identity/v3/role_assignments?group.id=06f307e7c0124171b92ab5dfc2d21f8d&scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba&role.id=b109a364aa914a98aee2e78986065def => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:20.125769 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da10b5b-384d-4e0e-b401-144605146d25 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.126055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da10b5b-384d-4e0e-b401-144605146d25 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.126603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da10b5b-384d-4e0e-b401-144605146d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.145343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9da10b5b-384d-4e0e-b401-144605146d25 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:20.147102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9da10b5b-384d-4e0e-b401-144605146d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:20.242800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 165/315] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:20.243952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da10b5b-384d-4e0e-b401-144605146d25 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.249744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:20.250168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dee97-c859-4feb-8e2a-51a45e4d19fd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.297278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f80abe-14f7-499e-9429-aa6285ba3df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.297905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80abe-14f7-499e-9429-aa6285ba3df7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.298185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80abe-14f7-499e-9429-aa6285ba3df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.298500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80abe-14f7-499e-9429-aa6285ba3df7 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.299186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f80abe-14f7-499e-9429-aa6285ba3df7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:20.299488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f80abe-14f7-499e-9429-aa6285ba3df7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:20.301137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f80abe-14f7-499e-9429-aa6285ba3df7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:20.315876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 151/316] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:20.317631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf78783-044d-4d08-a336-f61f52837832 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.318233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf78783-044d-4d08-a336-f61f52837832 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.318709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf78783-044d-4d08-a336-f61f52837832 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.327751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 152/317] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:20.328889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf78783-044d-4d08-a336-f61f52837832 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.392262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da10b5b-384d-4e0e-b401-144605146d25 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.441129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245a385-9b89-4ddd-8bce-a8f9dc820829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.441748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:20.442515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a385-9b89-4ddd-8bce-a8f9dc820829 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.442754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a385-9b89-4ddd-8bce-a8f9dc820829 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.442968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a385-9b89-4ddd-8bce-a8f9dc820829 None admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.445621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a385-9b89-4ddd-8bce-a8f9dc820829 None admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:20.447070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a385-9b89-4ddd-8bce-a8f9dc820829 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:20.448788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 166/318] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:20.458516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c29c-44ac-4f42-93f4-fe9afb20178d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.458516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c29c-44ac-4f42-93f4-fe9afb20178d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.458516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c29c-44ac-4f42-93f4-fe9afb20178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.477523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6ff3c29c-44ac-4f42-93f4-fe9afb20178d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:20.478966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6ff3c29c-44ac-4f42-93f4-fe9afb20178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:20.495236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf78783-044d-4d08-a336-f61f52837832 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.539471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.540220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.540574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.540915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.557574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:20.562662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 167/319] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:20.564862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c29c-44ac-4f42-93f4-fe9afb20178d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.628588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:20.631125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:20.702446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c29c-44ac-4f42-93f4-fe9afb20178d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.702999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:20.747131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 153/320] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:20.748227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.753173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.753980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.754307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.754582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.756108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] RBAC: Adding query filter params (group.id=06f307e7c0124171b92ab5dfc2d21f8d, role.id=a2f657dd4fab43288cb9e84eb6d94d51, scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:20.756230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:20.758212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:20.765229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 168/321] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/role_assignments?group.id=06f307e7c0124171b92ab5dfc2d21f8d&scope.project.id=3fa6e65e21f64df995b975bfb4ebdbba&role.id=a2f657dd4fab43288cb9e84eb6d94d51 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:20.766201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.842553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:20.879480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27a55b-18ee-4a02-a8a7-bf083d33da24 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.902482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959df83f-bab5-416a-bb49-9a83862fb0d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:20.930874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.931930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.932388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.932850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.946661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:20.947310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:20.947587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:20.947888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:20.948847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:20.949061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:20.950512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:20.952382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 169/322] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:20.953054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:20.953827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:20.954804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:20.955599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 154/323] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:20.961543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:20.994904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.080191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffdbcc-d7da-4162-b9c2-3e7b495cd85a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.090555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550240e-05de-4283-8454-98a2ddbd5fa8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.113318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.120477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.121107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.121387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.121660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.127660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.128306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.128588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.128866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.129529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:21.129827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.131446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.135161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:21.140546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:21.141427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:21.142209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 170/324] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:21.142856 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.145680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 155/325] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:21.149782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.251004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.265521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c42b0-34f0-4e27-aa0b-648a31f125a3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.266753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ea34a-7f4f-4cab-bb3d-bc33ab68c0b7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.304747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.305619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.305898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.306166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.307957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.309398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.310766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.311204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 171/326] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:21.311386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.311670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.311770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.311977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.312644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:21.312948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.314331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.325138 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 156/327] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:21.326088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.384253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.425223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3c2a0-b5e2-4081-8b42-80d2c82495b5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.434122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44274781-9658-4e8d-87b1-52021933e17f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.457558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.458151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.458423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.458712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.466933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.467532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.467842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.468119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.468851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] RBAC: Adding query filter params (group.id=06f307e7c0124171b92ab5dfc2d21f8d, role.id=b109a364aa914a98aee2e78986065def, scope.project.id=72927d62c8b04432867a524b73620dcb) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:21.469159 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.470592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.476968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 157/328] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/role_assignments?group.id=06f307e7c0124171b92ab5dfc2d21f8d&scope.project.id=72927d62c8b04432867a524b73620dcb&role.id=b109a364aa914a98aee2e78986065def => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:21.477842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.495665 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.535023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.536520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.594169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636def2-b93f-4a79-b353-5fd87c354d0e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.599079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.634361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.635088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.635371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.635913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.646773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 172/329] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:21.652880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.718802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.719944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.736862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.773636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923e490-57c2-42e6-8c22-7e5d4add1c30 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.817202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.819736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.819736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.819736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] PATH_INFO: `/v3/projects/72927d62c8b04432867a524b73620dcb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.820662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] RBAC: Authorizing `identity:get_project(project_id=72927d62c8b04432867a524b73620dcb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.822109 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.824360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 173/330] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/projects/72927d62c8b04432867a524b73620dcb => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:21.829477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.834379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 158/331] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:21.840232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.863516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.939511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8515e13-1d90-4dcd-98a7-32532f7c7454 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.946478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e0df-81ee-4bcc-9e1a-f1a39b002406 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:21.959907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:21.972957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.973584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.973862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.974134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] PATH_INFO: `/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.975858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] RBAC: Authorizing `identity:get_group(group_id=06f307e7c0124171b92ab5dfc2d21f8d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.977278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.978874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 174/332] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/groups/06f307e7c0124171b92ab5dfc2d21f8d => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:21.983139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:21.983139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:21.983139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:21.983139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] PATH_INFO: `/v3/projects/b00b7a3900004d669310a34e206f2fc2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:21.983464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] RBAC: Authorizing `identity:get_project(project_id=b00b7a3900004d669310a34e206f2fc2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:21.983464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:21.984705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:21.989473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 159/333] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/projects/b00b7a3900004d669310a34e206f2fc2 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:21.991118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:22.082594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:22.088811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb993f4-28e0-4b80-8f81-d93bb321c6c4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:22.092169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216824dd-f7ae-4d44-803d-6f45a70cdc74 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:22.123962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199f06e-b95e-4d4b-ba47-948005f7a919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:22.124565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199f06e-b95e-4d4b-ba47-948005f7a919 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:22.124850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199f06e-b95e-4d4b-ba47-948005f7a919 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:22.125121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199f06e-b95e-4d4b-ba47-948005f7a919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:22.125833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199f06e-b95e-4d4b-ba47-948005f7a919 None admin] RBAC: Adding query filter params (group.id=06f307e7c0124171b92ab5dfc2d21f8d, role.id=a2f657dd4fab43288cb9e84eb6d94d51, scope.project.id=72927d62c8b04432867a524b73620dcb) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:22.126144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199f06e-b95e-4d4b-ba47-948005f7a919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:22.126626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:22.127247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:22.127522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:22.127607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199f06e-b95e-4d4b-ba47-948005f7a919 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:22.127805 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] PATH_INFO: `/v3/groups/7e3971becff64204b5dc4c3630c85271` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:22.128891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] RBAC: Authorizing `identity:get_group(group_id=7e3971becff64204b5dc4c3630c85271)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:22.131061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:22.132664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 160/334] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/groups/7e3971becff64204b5dc4c3630c85271 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:22.133500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 175/335] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 26 10:50:21 2022] GET /identity/v3/role_assignments?group.id=06f307e7c0124171b92ab5dfc2d21f8d&scope.project.id=72927d62c8b04432867a524b73620dcb&role.id=a2f657dd4fab43288cb9e84eb6d94d51 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:22.138758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:22.194129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:22.247556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab8efe-997d-4ebf-b87f-93015b7de103 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:22.283774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51fabb-b16f-4900-9048-10ad90d6877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:22.284505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51fabb-b16f-4900-9048-10ad90d6877d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:22.284814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51fabb-b16f-4900-9048-10ad90d6877d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:22.285090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51fabb-b16f-4900-9048-10ad90d6877d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:22.285834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51fabb-b16f-4900-9048-10ad90d6877d None admin] RBAC: Adding query filter params (group.id=7e3971becff64204b5dc4c3630c85271, role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=b00b7a3900004d669310a34e206f2fc2) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:22.286146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51fabb-b16f-4900-9048-10ad90d6877d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:22.288441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51fabb-b16f-4900-9048-10ad90d6877d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:22.294938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 161/336] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 26 10:50:22 2022] GET /identity/v3/role_assignments?group.id=7e3971becff64204b5dc4c3630c85271&scope.project.id=b00b7a3900004d669310a34e206f2fc2&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:24.835464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350843b-afc7-48bb-87be-2b75c7cc3dee None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:24.835998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350843b-afc7-48bb-87be-2b75c7cc3dee None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:24.836073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350843b-afc7-48bb-87be-2b75c7cc3dee None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:24.837664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 176/337] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:24.842625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552c72-e958-4daf-860a-684275272971 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:24.843035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552c72-e958-4daf-860a-684275272971 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:24.843202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4552c72-e958-4daf-860a-684275272971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:24.866223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c4552c72-e958-4daf-860a-684275272971 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:24.867418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c4552c72-e958-4daf-860a-684275272971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:24.962035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 162/338] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:25.016850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b22de-b8ed-4f2b-9dbd-f65952f3f90d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.017167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b22de-b8ed-4f2b-9dbd-f65952f3f90d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.017410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b22de-b8ed-4f2b-9dbd-f65952f3f90d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.019014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 177/339] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:25.035548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2006e6a-e51f-4071-a0f1-1299b6633acb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.035774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2006e6a-e51f-4071-a0f1-1299b6633acb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.036072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2006e6a-e51f-4071-a0f1-1299b6633acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.064414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b2006e6a-e51f-4071-a0f1-1299b6633acb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:25.065642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b2006e6a-e51f-4071-a0f1-1299b6633acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:25.124346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:25.162778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 163/340] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:25.451745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b22de-b8ed-4f2b-9dbd-f65952f3f90d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:25.564165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e41f-c8e6-4c69-8474-fdbe4ed01bfa None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.564543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e41f-c8e6-4c69-8474-fdbe4ed01bfa None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.564732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e41f-c8e6-4c69-8474-fdbe4ed01bfa None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.566308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 164/341] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:25.571740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff96162-3c58-4b54-986e-261e7843a347 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.572058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff96162-3c58-4b54-986e-261e7843a347 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.572333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff96162-3c58-4b54-986e-261e7843a347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.586162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b22de-b8ed-4f2b-9dbd-f65952f3f90d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:25.592231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1ff96162-3c58-4b54-986e-261e7843a347 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:25.593420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1ff96162-3c58-4b54-986e-261e7843a347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:25.629218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:25.630020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.630437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.630717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.643697 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:25.645593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:25.662059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:25.663125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:25.664220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:25.664220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:25.664220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:25.664220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 178/342] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:25.665385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:25.668252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 165/343] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:25.668252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a715ff5-ac1b-414d-bc0a-db0ce3183b35 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.668252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a715ff5-ac1b-414d-bc0a-db0ce3183b35 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.668252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a715ff5-ac1b-414d-bc0a-db0ce3183b35 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.668988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 166/344] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:25.669690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a715ff5-ac1b-414d-bc0a-db0ce3183b35 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:25.695757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:25.798843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a715ff5-ac1b-414d-bc0a-db0ce3183b35 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:25.802107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:25.805437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe115a-5c00-4220-bc7c-0fcccabaf60c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:25.843196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:25.843946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.844098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.844365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.848842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab28e4f-80d7-40bf-bb50-5e154263b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:25.851837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab28e4f-80d7-40bf-bb50-5e154263b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.851837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab28e4f-80d7-40bf-bb50-5e154263b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.851837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab28e4f-80d7-40bf-bb50-5e154263b1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.851837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab28e4f-80d7-40bf-bb50-5e154263b1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:25.851837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab28e4f-80d7-40bf-bb50-5e154263b1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:25.853521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab28e4f-80d7-40bf-bb50-5e154263b1f0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:25.856350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:25.857874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:25.873334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:25.875065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:25.877149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:25.877149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:25.877149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:25.877510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 167/345] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:25.881159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 179/346] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:25.881766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad387b0-27af-44ac-bf3c-2a2b9c7a96ee None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:25.881984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:25.882153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad387b0-27af-44ac-bf3c-2a2b9c7a96ee None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:25.883815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad387b0-27af-44ac-bf3c-2a2b9c7a96ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:25.902154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0ad387b0-27af-44ac-bf3c-2a2b9c7a96ee None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:25.903329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0ad387b0-27af-44ac-bf3c-2a2b9c7a96ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:25.963960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:25.974867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 168/347] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:25.976672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad387b0-27af-44ac-bf3c-2a2b9c7a96ee None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.003080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cdb5c-a4cc-4bb6-ad3d-9481173bc2fa None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:26.047221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:26.047898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.048182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.048455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.049157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:26.050873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:26.095353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:26.102024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad387b0-27af-44ac-bf3c-2a2b9c7a96ee None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:26.136258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 180/348] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:25 2022] POST /identity/v3/users => generated 246 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:26.137212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.144256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:26.145253 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.147100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.147630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.148555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:26.149113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:26.151245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:26.167330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 169/349] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:26.183519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.312466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cdb37-009b-48d1-bd8d-e9c25e50d78b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:26.335094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:26.349167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:26.354362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.354362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.354362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.354622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902d98b-5bdf-47fb-9ea3-552274d15ed1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:26.363682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:26.365797 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:26.380861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:26.381990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:26.382998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:26.382998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:26.382998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:26.384585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 181/350] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:26.386134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.403164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:26.403810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.404090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.404366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.405021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:26.406511 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:26.491744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408ea32-c7a3-4e78-a5c6-7f0f916325c9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:26.516371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 170/351] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:26 2022] POST /identity/v3/users => generated 248 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:26.517397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.558770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:26.559705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.560015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.560285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.579398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:26.582089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:26.583983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:26.618733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:26.619930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:26.620759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:26.620759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:26.620759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:26.620759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 182/352] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:26.626722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.660854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:26.710307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192827b-2373-4f47-b423-4c8f9a2f8874 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:26.761553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf94bdf-00f1-42bd-9ae0-bc07162ebbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:26.762173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf94bdf-00f1-42bd-9ae0-bc07162ebbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.762453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf94bdf-00f1-42bd-9ae0-bc07162ebbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.762729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf94bdf-00f1-42bd-9ae0-bc07162ebbd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.763375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf94bdf-00f1-42bd-9ae0-bc07162ebbd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:26.763681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf94bdf-00f1-42bd-9ae0-bc07162ebbd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:26.765889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf94bdf-00f1-42bd-9ae0-bc07162ebbd8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:26.776238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6797b1f-f132-4437-9a79-d15fae636b7a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:26.781359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 171/353] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:26.782686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e8c6e-6e12-4302-b975-d6de4a1a545f None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.782973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e8c6e-6e12-4302-b975-d6de4a1a545f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.783239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e8c6e-6e12-4302-b975-d6de4a1a545f None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.784478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 172/354] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:26.790116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6016-eb03-49b8-a704-4cd1725842a3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.790397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6016-eb03-49b8-a704-4cd1725842a3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.790669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6016-eb03-49b8-a704-4cd1725842a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.809973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f71d6016-eb03-49b8-a704-4cd1725842a3 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:26.811070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f71d6016-eb03-49b8-a704-4cd1725842a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:26.817181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:26.817852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:26.818132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:26.818404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:26.820259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:26.820565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:26.822314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:26.836869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 183/355] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:26.837633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.883099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:26.898104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 173/356] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:26.899571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6016-eb03-49b8-a704-4cd1725842a3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:26.944544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:26.964945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390639a0-586f-4065-bfa7-5d07e6222b83 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:27.008098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:27.009166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:27.009638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:27.010171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:27.011290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:27.012963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:27.027866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6016-eb03-49b8-a704-4cd1725842a3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:27.067990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:27.068685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:27.069013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:27.069351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:27.070187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:27.071705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:27.130017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 184/357] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:26 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:27.153702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 174/358] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:26 2022] POST /identity/v3/users => generated 249 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:27.244013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:27.390197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b824c-9b4f-4277-ba31-36d5133deba6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:27.427211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:27.427844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:27.428165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:27.428441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:27.429099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:27.430503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:27.468733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.sql.core [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 26 10:50:27.468733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 26 10:50:27.468733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [parameters: {'id': '3198134fabf140d08cd2eabfd29383e5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 26 10:50:27.468733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81607) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:50:27.476342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return ref.to_dict() Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:50:27.477195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application yield resource Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise self.value Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application session.commit() Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 10:50:27.478507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.flush() Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application compat.raise_( Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 10:50:27.479571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:50:27.480573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:50:27.481375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [parameters: {'id': '3198134fabf140d08cd2eabfd29383e5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:27.482903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 26 10:50:27.485025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:27.487016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 185/359] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 26 10:50:27 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:27.487149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:27.570516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:27.619147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18087980-ea11-4a50-9fd6-77a8452dc853 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:27.656109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fdb45-2cae-403f-a2f5-7840157e02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:27.656671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fdb45-2cae-403f-a2f5-7840157e02e6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:27.656953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fdb45-2cae-403f-a2f5-7840157e02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:27.657227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fdb45-2cae-403f-a2f5-7840157e02e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:27.668692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fdb45-2cae-403f-a2f5-7840157e02e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-5b2fdb45-2cae-403f-a2f5-7840157e02e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:27.674077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:27.676300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:27.678272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 175/360] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:50:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:27.681725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:27.686739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:27.803481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f404005-eea4-4234-9a4d-179dabebca35 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:27.838501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5170-130c-4b7e-b8d2-db8280896fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:27.839189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5170-130c-4b7e-b8d2-db8280896fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:27.839471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5170-130c-4b7e-b8d2-db8280896fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:27.839745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5170-130c-4b7e-b8d2-db8280896fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:27.840483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5170-130c-4b7e-b8d2-db8280896fc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:27.840791 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5170-130c-4b7e-b8d2-db8280896fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:27.842155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5170-130c-4b7e-b8d2-db8280896fc5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:27.854751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 186/361] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:28.947482 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2bd56-08c6-4e1e-9521-2009658203f2 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:28.948050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2bd56-08c6-4e1e-9521-2009658203f2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:28.948050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2bd56-08c6-4e1e-9521-2009658203f2 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:28.949656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 176/362] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:28.961028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6736803-452c-4cb8-ae10-6b12650d3307 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:28.961438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6736803-452c-4cb8-ae10-6b12650d3307 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:28.961648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6736803-452c-4cb8-ae10-6b12650d3307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:28.987719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a6736803-452c-4cb8-ae10-6b12650d3307 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:28.987719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a6736803-452c-4cb8-ae10-6b12650d3307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:29.059993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:29.068581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 187/363] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:29.208684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe2355-4db0-4a56-a335-a97342eb4c1c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:29.208994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe2355-4db0-4a56-a335-a97342eb4c1c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:29.209225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe2355-4db0-4a56-a335-a97342eb4c1c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:29.210739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 177/364] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:29.216150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8c7f-9781-4adb-b103-9dc994a2d9d8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:29.216498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8c7f-9781-4adb-b103-9dc994a2d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:29.216783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8c7f-9781-4adb-b103-9dc994a2d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:29.236370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0a8c8c7f-9781-4adb-b103-9dc994a2d9d8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:29.237555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0a8c8c7f-9781-4adb-b103-9dc994a2d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:29.367518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 188/365] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:29.394180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe2355-4db0-4a56-a335-a97342eb4c1c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:29.537602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe2355-4db0-4a56-a335-a97342eb4c1c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:29.563476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:29.585535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:29.586236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:29.586525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:29.586795 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:29.604918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:29.612671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:29.613622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:29.614432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 178/366] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:29.621580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c8c7f-9781-4adb-b103-9dc994a2d9d8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:29.720862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:29.732682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c8c7f-9781-4adb-b103-9dc994a2d9d8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:29.762308 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:29.770873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2af6cd-da9a-4876-b10d-d5cb1628b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:29.779113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af6cd-da9a-4876-b10d-d5cb1628b76d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:29.779113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af6cd-da9a-4876-b10d-d5cb1628b76d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:29.779113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af6cd-da9a-4876-b10d-d5cb1628b76d None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:29.779113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2af6cd-da9a-4876-b10d-d5cb1628b76d None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:29.779113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2af6cd-da9a-4876-b10d-d5cb1628b76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:29.779113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2af6cd-da9a-4876-b10d-d5cb1628b76d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:29.784096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 189/367] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:29.785544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b7f3c-140f-4577-ae3e-623c00e7f7ee None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:29.785879 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b7f3c-140f-4577-ae3e-623c00e7f7ee None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:29.786481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b7f3c-140f-4577-ae3e-623c00e7f7ee None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:29.787746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 190/368] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:29.793271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c77fa-3f87-4326-b54c-c66be4a78393 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:29.793556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c77fa-3f87-4326-b54c-c66be4a78393 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:29.793893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c77fa-3f87-4326-b54c-c66be4a78393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:29.815142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7d3c77fa-3f87-4326-b54c-c66be4a78393 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:29.817026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7d3c77fa-3f87-4326-b54c-c66be4a78393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:29.857717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2158a-b8ee-4056-9815-4ab0bfe4f9a2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:29.892780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:29.893439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:29.893716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:29.893985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:29.911810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 191/369] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:29.911965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:29.912852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:29.913681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 179/370] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:29.914423 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:29.919033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c77fa-3f87-4326-b54c-c66be4a78393 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:29.967334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.093374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24e75a-23f2-4b83-8fc6-2d4077754b1a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.108480 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c77fa-3f87-4326-b54c-c66be4a78393 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.137101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.151919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e1bd0-96b0-4fab-847e-86fca21d44e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.152555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e1bd0-96b0-4fab-847e-86fca21d44e1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.152832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e1bd0-96b0-4fab-847e-86fca21d44e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.157921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e1bd0-96b0-4fab-847e-86fca21d44e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.158506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669cc01-37e4-4974-994b-3195a3fbeedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.159116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669cc01-37e4-4974-994b-3195a3fbeedd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.159395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669cc01-37e4-4974-994b-3195a3fbeedd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.159776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669cc01-37e4-4974-994b-3195a3fbeedd None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.160473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669cc01-37e4-4974-994b-3195a3fbeedd None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:30.160774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669cc01-37e4-4974-994b-3195a3fbeedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.162336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669cc01-37e4-4974-994b-3195a3fbeedd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.164076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e1bd0-96b0-4fab-847e-86fca21d44e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.165532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e1bd0-96b0-4fab-847e-86fca21d44e1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.171641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 192/371] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:30.173003 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0134e-7a7f-424e-97f7-3efbb46cd23c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.173940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0134e-7a7f-424e-97f7-3efbb46cd23c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.174221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0134e-7a7f-424e-97f7-3efbb46cd23c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.179905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 193/372] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:30.180884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0134e-7a7f-424e-97f7-3efbb46cd23c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3c4e1bd0-96b0-4fab-847e-86fca21d44e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:30.182232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:30.183073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:30.183953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:30.183953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:30.183953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:30.187869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 180/373] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:30.189133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2c9c-e5c9-4b0d-b31b-54fe0bc6c1d2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.189415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2c9c-e5c9-4b0d-b31b-54fe0bc6c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.189689 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2c9c-e5c9-4b0d-b31b-54fe0bc6c1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.213235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5bba2c9c-e5c9-4b0d-b31b-54fe0bc6c1d2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:30.214458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5bba2c9c-e5c9-4b0d-b31b-54fe0bc6c1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:30.293024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 181/374] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:30.293876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba2c9c-e5c9-4b0d-b31b-54fe0bc6c1d2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.308489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0134e-7a7f-424e-97f7-3efbb46cd23c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.322617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.348936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.349494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.349850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.350126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.362762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.364808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:30.381975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:30.382950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:30.384489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:30.384489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:30.384489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:30.384489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 194/375] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:30.384489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.419913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.429418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba2c9c-e5c9-4b0d-b31b-54fe0bc6c1d2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.472848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.474366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.474981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.475398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.476939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:30.477387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.479210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.494094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 182/376] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:30.495224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.501089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5310b-580e-45bb-83de-25d31d2d7bce None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.541755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.542735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.543214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.543687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.557516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:30.563909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:30.564787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:30.565596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:30.573482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 195/377] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:30.573482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.593641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.616877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57127-087d-4242-920d-04e185b98bb6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.651468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.652256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.652533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.652801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.653433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:30.653739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.655110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.668029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 183/378] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:30.669116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.674953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.684897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f816-0716-49d0-88aa-37152f2a9a4b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.723143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.724174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.724632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.725074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.733377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.734878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:30.748632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:30.749553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 196/379] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:30.755510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.783886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529ef4a-e49b-400c-9a84-5797d7d369c8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.820002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.820609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.820886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.821156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.821809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:30.822109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.823490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.828466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.832234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 184/380] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:30.833025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.862904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f608f4-f3d2-4b88-b3af-73c5a8c4cdd4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.895429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.896526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.896898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.897168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.906832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:30.912595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:30.913602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 197/381] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:30.914638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:30.928607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:30.939113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abdbba-95c7-481a-af49-2183ee872563 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:30.973958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c299cc-36f8-41e6-b087-3cc0b21dad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:30.974730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c299cc-36f8-41e6-b087-3cc0b21dad7b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.975011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c299cc-36f8-41e6-b087-3cc0b21dad7b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:30.975283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c299cc-36f8-41e6-b087-3cc0b21dad7b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:30.982782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c299cc-36f8-41e6-b087-3cc0b21dad7b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:30.984286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c299cc-36f8-41e6-b087-3cc0b21dad7b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-46c299cc-36f8-41e6-b087-3cc0b21dad7b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:30.996757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:30.997688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:30.998582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 185/382] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:30.999471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307b69-2775-4957-a7cf-37f0ac81e122 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:30.999755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307b69-2775-4957-a7cf-37f0ac81e122 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.000049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307b69-2775-4957-a7cf-37f0ac81e122 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.001308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 186/383] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:31.002049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f307b69-2775-4957-a7cf-37f0ac81e122 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.018435 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1dbe7-ac5d-44c7-98e2-0c7d1c472c1f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.055319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.055956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.056237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.056509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.057228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:31.057532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.058975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.064409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.096179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 198/384] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 26 10:50:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:31.097014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.106977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f307b69-2775-4957-a7cf-37f0ac81e122 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.140696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.141324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.141597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.141864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.152838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.154233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:31.168147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:31.169222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:31.170292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:31.170292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.170292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:31.170467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.170879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 187/385] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:31.170879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.202447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b863c91-1ea6-4bd0-b999-01f9c930b7ff None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.237086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c2076-b002-4b15-8b13-e5f99abd3de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.237688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c2076-b002-4b15-8b13-e5f99abd3de0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.237968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c2076-b002-4b15-8b13-e5f99abd3de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.238249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c2076-b002-4b15-8b13-e5f99abd3de0 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.238910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c2076-b002-4b15-8b13-e5f99abd3de0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:31.239215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c2076-b002-4b15-8b13-e5f99abd3de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.240686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c2076-b002-4b15-8b13-e5f99abd3de0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.248559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 199/386] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:31.254217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2a3a7-934c-4c3a-88d7-d7ca71f0d03a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.254504 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2a3a7-934c-4c3a-88d7-d7ca71f0d03a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.254787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2a3a7-934c-4c3a-88d7-d7ca71f0d03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.270087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6885b-835b-45e3-a96f-ba98819e466d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.273773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6db2a3a7-934c-4c3a-88d7-d7ca71f0d03a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:31.274994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6db2a3a7-934c-4c3a-88d7-d7ca71f0d03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:31.284062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.305532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.306134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.306405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.306668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.307385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:31.307680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.309128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.340711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 200/387] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:31.341554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2a3a7-934c-4c3a-88d7-d7ca71f0d03a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.353263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 188/388] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:31.354177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.413234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.453722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2a3a7-934c-4c3a-88d7-d7ca71f0d03a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.459333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d46ba-627e-4646-b03d-2252a6c7d491 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.490761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.491389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.491666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.491978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.493069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.493675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.493956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.494228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.494879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:31.495187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.496600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.502517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.504121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.512369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 189/389] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:31.512369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:31.522020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:31.522898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:31.523757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:31.523757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.523757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:31.523757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 201/390] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:31.523988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.549719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.622559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afb7ae-122d-478d-b9c3-39b178ee91e1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.631934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba6b6-55d8-4895-bb14-9ba9ba048cb6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.645415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.656357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.657008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.657280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.657552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.665461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.666023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.666403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.666700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.668555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.669996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.679188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.680684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:31.684307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:31.685181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:31.686029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:31.686029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.686029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:31.686029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 190/391] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:31.686249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:31.695611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:31.696539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:31.697408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:31.697408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:31.697408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:31.697408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 202/392] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:31.697619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.774812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.793165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081309c-8788-4009-bf84-0678181b5df3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.796285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9d85e-2bfd-4b90-89eb-b9b9e28e5a0e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.827079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.827697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.828012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.828285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.829448 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.830052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.830318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.830584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.831208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:31.831503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.832880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.835209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.836676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.844679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 203/393] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:31.846619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:31.848202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:31.849922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 191/394] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:31.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:31.876681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:31.948628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853661fd-5d78-4480-b3ad-40d2b9e70efc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.950763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbb88a-31ac-488d-b04c-7f63ef592a53 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:31.981455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.982061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.982406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.982677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.983299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:31.983597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.984613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:31.984991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.985233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:31.985511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:31.985781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:31.996306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:31.997357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 192/395] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:31.997738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:31.998167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.000963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:32.012673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:32.015264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:32.016322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:32.016322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:32.016322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:32.016322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 204/396] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:32.016322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.099345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.100173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0020d7a-5928-4e42-9365-4c6175baecdd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.120164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fa62c-e12f-489b-8d17-be5d7c55ea1a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.134251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.134851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.135133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.135652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.136325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:32.136629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.138017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.149800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 193/397] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:32.152672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.154155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.154786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.155244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.155518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.162422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.163930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:32.177344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:32.178804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 205/398] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:32.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.221203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.260306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c200-0f71-40d5-8ed1-8a9e8641a73c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.281502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9617d-8d75-41ae-bf13-a267163f2287 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.293588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.294220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.294495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.294772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.295486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:32.295804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.297241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.314619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 194/399] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:32.314833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.315416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.315512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.315852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.316067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.328962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.330410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.332190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:32.342223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:32.347963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 206/400] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:32.348970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.417549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87459841-f4fc-4ad7-946b-6082f6b77ef4 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.437170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.444295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54398226-e40c-4697-9b10-21d0a311491c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.450948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.451586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.451985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.452256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.452880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:32.453177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.454534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.466790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 195/401] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:32.467594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.482538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.483095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.483371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.483643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.490698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.492157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:32.503132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:32.504156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:32.505591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 207/402] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:32.506133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.562677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.569339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82f114-bb8e-4f23-a078-ea13541fd51c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.602698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.603295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.603572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.603860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.604439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1da22a-0c33-493d-8096-2eb280fc8cb9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.604543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:32.604830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.606249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.623335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 196/403] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:32.624169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.638562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.639196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.639472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.639744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.651280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.652748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.665750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:32.667983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:32.669072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:32.669948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:32.669948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:32.669948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:32.669948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 208/404] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:32.669948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.725152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335097d-2d04-481b-81d3-22624050c4f7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.758112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.758644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.758925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.759194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.759944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:32.760247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.761944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.768133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03aa36-f4b0-4d78-a276-aca8f7fe0907 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.778858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.779245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 197/405] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:32.780075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.803597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.804237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.804518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.804822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.805675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.807454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.875655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:32.879374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22555d0d-e05f-4c60-9f2c-9508fa549a34 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:32.885948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 209/406] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:32 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:32.886868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:32.919011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:32.919640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:32.919952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:32.920221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:32.921103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:32.921459 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:32.922961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:32.941236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 198/407] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:32.942071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.003588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47836c-24b1-4764-939b-f4ef842a3c7c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.052221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.052829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.053233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.053513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/c3d05bdc4c9b4f24b1d58ef157b32ae0/roles/7f166188476b48158de248c89d0b9144` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.066605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d008e5c-b388-4c16-9ee1-a7725d3eca01 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.069042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=c3d05bdc4c9b4f24b1d58ef157b32ae0, role_id=7f166188476b48158de248c89d0b9144)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.071138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.103588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.104236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.104494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.104762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.105469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:33.105762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.107133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.119088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 199/408] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:33.119727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.138890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:33.155821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 210/409] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:32 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/c3d05bdc4c9b4f24b1d58ef157b32ae0/roles/7f166188476b48158de248c89d0b9144 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:33.155821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.178367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:33.241029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1f6a-9cb4-41c0-915f-792fc69217a5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.281393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.282162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.282509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.282839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.287048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c07d0-07b3-4b73-9d3c-a4711ef0d05a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.294864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.296445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:33.313272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:33.319149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:33.320252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:33.320252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:33.320252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:33.320252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 200/410] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:33.320252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.336563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.337348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.337700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.338436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/9627e6a9e86e43b694f723bd4109460e/roles/7f166188476b48158de248c89d0b9144` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.361227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=9627e6a9e86e43b694f723bd4109460e, role_id=7f166188476b48158de248c89d0b9144)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.363467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.399433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:33.430788 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89d411-49c6-4a1b-b653-b0e552c3beba None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.444987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 211/411] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:33 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/9627e6a9e86e43b694f723bd4109460e/roles/7f166188476b48158de248c89d0b9144 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:33.445603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.480008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.481998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.484527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.485071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.502405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.507578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:33.537888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:33.538832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:33.544741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 201/412] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:33.561003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.577277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:33.622674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d26822-b3f7-4550-b586-315b31f9996e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.678994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.679627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.679936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.680213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.680868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:33.681176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.682566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.702358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 212/413] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:33.709378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.749265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece22ad-7c16-4ea3-be5f-7294a5d94087 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.781557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:33.787206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.790262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.790560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.790835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.795149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:33.795149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.795149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.807409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 202/414] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:33.813788 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:33.884409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a27e7-b1b8-476b-8cbe-cdd665f32cc7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:33.939470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:33.944727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:33.945602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:33.948134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:33.948679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:33.962722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:33.965690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:33.988241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:33.989520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:33.990408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:33.998540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 213/415] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:34.005557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:34.008761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4123050-6a40-41b7-951d-e3d804a73a79 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:34.092754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbce10-c8f7-4fa0-8e58-50059f0127fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:34.094184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbce10-c8f7-4fa0-8e58-50059f0127fd None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:34.094668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbce10-c8f7-4fa0-8e58-50059f0127fd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:34.095138 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbce10-c8f7-4fa0-8e58-50059f0127fd None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/37b474a596cf42efb5e0e5a2a179735e/roles/7f166188476b48158de248c89d0b9144` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:34.125133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbce10-c8f7-4fa0-8e58-50059f0127fd None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=37b474a596cf42efb5e0e5a2a179735e, role_id=7f166188476b48158de248c89d0b9144)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:34.126929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbce10-c8f7-4fa0-8e58-50059f0127fd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:34.155353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958697f5-15ca-434f-b90a-e0152e958746 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:34.196000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62526793-605c-442c-a36e-612e01813a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:34.196673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62526793-605c-442c-a36e-612e01813a21 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:34.196912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62526793-605c-442c-a36e-612e01813a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:34.197132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62526793-605c-442c-a36e-612e01813a21 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:34.197818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62526793-605c-442c-a36e-612e01813a21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:34.198123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62526793-605c-442c-a36e-612e01813a21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:34.199559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62526793-605c-442c-a36e-612e01813a21 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:34.218705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 214/416] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:34.225622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62526793-605c-442c-a36e-612e01813a21 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:34.246271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 203/417] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:33 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/37b474a596cf42efb5e0e5a2a179735e/roles/7f166188476b48158de248c89d0b9144 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:50:34.279023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:34.338859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62526793-605c-442c-a36e-612e01813a21 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:34.377239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e597cb-a823-44f3-bb0c-fd2f1dde59d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:34.377905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e597cb-a823-44f3-bb0c-fd2f1dde59d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:34.378181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e597cb-a823-44f3-bb0c-fd2f1dde59d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:34.378449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e597cb-a823-44f3-bb0c-fd2f1dde59d9 None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/ce0cb1d976784ea6af783fd193ddac20/roles/7f166188476b48158de248c89d0b9144` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:34.396253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e597cb-a823-44f3-bb0c-fd2f1dde59d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=ce0cb1d976784ea6af783fd193ddac20, role_id=7f166188476b48158de248c89d0b9144)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:34.398120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e597cb-a823-44f3-bb0c-fd2f1dde59d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:34.475885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 215/418] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:34 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/ce0cb1d976784ea6af783fd193ddac20/roles/7f166188476b48158de248c89d0b9144 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:50:35.213800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5022a4d-5fc3-4834-bddb-c74183145d11 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:35.214820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5022a4d-5fc3-4834-bddb-c74183145d11 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:35.215296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5022a4d-5fc3-4834-bddb-c74183145d11 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:35.216879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 204/419] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:35.222273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6aac0-c47e-46f7-b3c1-7926978e08de None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:35.222680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6aac0-c47e-46f7-b3c1-7926978e08de None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:35.222850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6aac0-c47e-46f7-b3c1-7926978e08de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:35.246187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-27c6aac0-c47e-46f7-b3c1-7926978e08de None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:35.247393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-27c6aac0-c47e-46f7-b3c1-7926978e08de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:35.334771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 216/420] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:35.594203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0362-36a2-46b6-bc8b-179c24eea30b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:35.594565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0362-36a2-46b6-bc8b-179c24eea30b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:35.595004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0362-36a2-46b6-bc8b-179c24eea30b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:35.596617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 205/421] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:35.602922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d004ef4-8c56-431c-974b-99b9aee394b0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:35.603201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d004ef4-8c56-431c-974b-99b9aee394b0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:35.603469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d004ef4-8c56-431c-974b-99b9aee394b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:35.628759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9d004ef4-8c56-431c-974b-99b9aee394b0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:35.630661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9d004ef4-8c56-431c-974b-99b9aee394b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:35.658070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:35.725839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 217/422] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:35.743479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0362-36a2-46b6-bc8b-179c24eea30b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:35.821792 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:35.906854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0362-36a2-46b6-bc8b-179c24eea30b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:35.956604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:35.957231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:35.957506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:35.958319 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:35.971529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:35.976676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:35.977555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:35.978346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 206/423] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:35.985497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d004ef4-8c56-431c-974b-99b9aee394b0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.113833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.136130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d004ef4-8c56-431c-974b-99b9aee394b0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:36.219268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:36.220404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.222963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.224256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.225884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:36.227635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:36.229240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:36.235871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:36.241194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2a5d8-543a-4b06-8221-581baecbd580 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:36.246148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 218/424] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:36.252639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.275387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:36.309266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:36.309901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.310178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.310449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.335308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:36.342693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:36.343738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:36.344607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 207/425] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:36.353447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.394654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91b27b-98e2-4a5e-9e91-aa875ddc9654 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:36.431176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:36.432011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.432392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.432763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.445079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:36.446685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:36.461470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:36.462412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:36.463264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:36.463264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:36.463264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:36.463264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 219/426] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:36.469429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.537941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d4520-394c-4aba-83ea-9e355fea592f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:36.557164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:36.575208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2758b-9d0a-438b-acf8-33418346a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:36.575836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2758b-9d0a-438b-acf8-33418346a62d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.576123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2758b-9d0a-438b-acf8-33418346a62d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.576397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2758b-9d0a-438b-acf8-33418346a62d None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.577055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2758b-9d0a-438b-acf8-33418346a62d None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:36.577526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2758b-9d0a-438b-acf8-33418346a62d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:36.579226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2758b-9d0a-438b-acf8-33418346a62d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:36.595246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 208/427] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:36.595246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00145-e4bc-43df-8aea-e685a0bf42b4 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.595246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00145-e4bc-43df-8aea-e685a0bf42b4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.595246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00145-e4bc-43df-8aea-e685a0bf42b4 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.595246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 209/428] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:36.598047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b00145-e4bc-43df-8aea-e685a0bf42b4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.602626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a620f-1ecb-4fde-a329-86bfb3efd395 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:36.626823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:36.642394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37deeae0-cb50-4556-ae9c-58d864cd0ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:36.643004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deeae0-cb50-4556-ae9c-58d864cd0ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.643283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deeae0-cb50-4556-ae9c-58d864cd0ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.643548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deeae0-cb50-4556-ae9c-58d864cd0ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.644224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37deeae0-cb50-4556-ae9c-58d864cd0ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:36.644527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37deeae0-cb50-4556-ae9c-58d864cd0ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:36.646415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37deeae0-cb50-4556-ae9c-58d864cd0ea7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:36.660026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 220/429] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:36.664973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c96d74-104c-4b38-b327-fac28f89fd70 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.664973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c96d74-104c-4b38-b327-fac28f89fd70 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.664973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c96d74-104c-4b38-b327-fac28f89fd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.683237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f7c96d74-104c-4b38-b327-fac28f89fd70 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:36.684492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f7c96d74-104c-4b38-b327-fac28f89fd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:36.722887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b00145-e4bc-43df-8aea-e685a0bf42b4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:36.760657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:36.761651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:36.762148 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:36.762611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:36.767853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 221/430] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:36.768999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c96d74-104c-4b38-b327-fac28f89fd70 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.788132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:36.789944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:36.815531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:36.816518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:36.818009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:36.818009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:36.818009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:36.820126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 210/431] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:36.828330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:36.893059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:36.976428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c96d74-104c-4b38-b327-fac28f89fd70 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:36.985028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:37.025998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf315e6-2bbb-48c2-9b42-99247cdf452d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.082401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b873b-c360-4569-a152-9dbe4c65fc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.083573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b873b-c360-4569-a152-9dbe4c65fc35 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.084012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b873b-c360-4569-a152-9dbe4c65fc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.084287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b873b-c360-4569-a152-9dbe4c65fc35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.089105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd38739-6ca9-4b82-aa9b-2ce2f222cd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.089721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd38739-6ca9-4b82-aa9b-2ce2f222cd7b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.090000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd38739-6ca9-4b82-aa9b-2ce2f222cd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.090358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd38739-6ca9-4b82-aa9b-2ce2f222cd7b None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.091013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd38739-6ca9-4b82-aa9b-2ce2f222cd7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:37.091313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd38739-6ca9-4b82-aa9b-2ce2f222cd7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.093426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd38739-6ca9-4b82-aa9b-2ce2f222cd7b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.096736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b873b-c360-4569-a152-9dbe4c65fc35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.099011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b873b-c360-4569-a152-9dbe4c65fc35 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.107150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 211/432] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:37.108450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647816b-dd18-4b19-aefe-0aed435a478d None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.108734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647816b-dd18-4b19-aefe-0aed435a478d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.109008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647816b-dd18-4b19-aefe-0aed435a478d None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e30b873b-c360-4569-a152-9dbe4c65fc35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:37.112998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:37.114176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:37.115180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:37.115676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 212/433] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:37.115676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647816b-dd18-4b19-aefe-0aed435a478d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.115818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 222/434] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:37.116622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22a88-a7a2-4304-a32a-523bfc379612 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.116903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22a88-a7a2-4304-a32a-523bfc379612 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.117172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22a88-a7a2-4304-a32a-523bfc379612 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.118557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 223/435] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:37.119137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc22a88-a7a2-4304-a32a-523bfc379612 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.216631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647816b-dd18-4b19-aefe-0aed435a478d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.226356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc22a88-a7a2-4304-a32a-523bfc379612 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.246272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:37.250273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5255d-d440-4b07-9745-1349353af959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.250894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5255d-d440-4b07-9745-1349353af959 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.251172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5255d-d440-4b07-9745-1349353af959 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.251442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5255d-d440-4b07-9745-1349353af959 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.253306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:37.261013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.261633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.266038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.266038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.266220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5255d-d440-4b07-9745-1349353af959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6eb5255d-d440-4b07-9745-1349353af959 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:37.267485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:37.268610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:37.274186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 213/436] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:37.274848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.274936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7541a9-313c-42ea-a111-d9857e80bbb5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.274936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7541a9-313c-42ea-a111-d9857e80bbb5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.274936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7541a9-313c-42ea-a111-d9857e80bbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.275106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:37.285689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:37.286708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:37.287622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 224/437] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:37.288222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.289860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5e7541a9-313c-42ea-a111-d9857e80bbb5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:37.291120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5e7541a9-313c-42ea-a111-d9857e80bbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:37.357842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 214/438] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:37.359152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de041-109a-4c18-bb39-8544fbe623ba None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.359439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de041-109a-4c18-bb39-8544fbe623ba None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.359712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de041-109a-4c18-bb39-8544fbe623ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.377990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6a5de041-109a-4c18-bb39-8544fbe623ba None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:37.379170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6a5de041-109a-4c18-bb39-8544fbe623ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:37.403218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3eca4-27a7-490a-a4ad-214c6b912e19 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.440394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.440997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.441273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.441537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.443444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:37.443925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.445655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.456010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 215/439] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:37.456010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5de041-109a-4c18-bb39-8544fbe623ba None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.478662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 225/440] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:37.478662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.526011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:37.532775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:37.596102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5de041-109a-4c18-bb39-8544fbe623ba None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.597252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb03bbe-1cd5-4580-bb61-2c823e21ae9b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.633826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.634519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.634867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.635200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.636304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:37.636637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.638111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.638635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.639240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.639518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.639828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.640462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:37.640760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.642140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.655605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 216/441] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:37.655605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.660060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 226/442] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:37.660288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.760218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66439f-99fe-470e-a785-6add9ebf94f5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.761358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:37.766975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f290c4d-2f15-401b-a3e7-6f41be97ca71 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.778227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:37.794093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.794710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.794982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.795248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.801165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.801815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.806838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.806838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.807023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.807370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.812723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:37.818141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:37.819228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 227/443] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:37.820433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:37.825481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:37.826569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:37.827565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:37.827565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:37.827565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:37.830382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 217/444] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:37.831204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.923874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675c07e-aa81-485a-908c-d3887695bd6f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.934164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3f747-e03d-45e1-95d0-00ccfd298c55 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:37.958410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.959028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.959308 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.959580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.968861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:37.969608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:37.969888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:37.969977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:37.970164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:37.971363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:37.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:37.986612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:37.987466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:37.987466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:37.987466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:37.987466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 228/445] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:37.987466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:37.998873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.008315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.043070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.044650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.087574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47abf9f-44ff-47b8-b85e-635c84670a77 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.120750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.121372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.121647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.121916 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.132369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:38.137838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:38.138746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:38.139545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 229/446] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:38.140253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.155622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 218/447] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:38.155622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.209839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.216733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.242761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabdf10-975e-4204-b320-a2ef5226cfda None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.249646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ec9c2-6d97-48d6-b67f-521a5fdea4ee None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.277908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.278501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.278775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.279041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.279684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:38.280080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.281427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.283073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.283674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.283984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.284255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.284902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:38.285187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.286641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.290064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 230/448] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:38.292656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.299641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 219/449] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:38.302677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.390795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eaf55-8cd2-4cba-9ac4-0fbdd0ab6457 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.400120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2557f9-74ab-4b1f-a1dc-0fc20e000514 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.418599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.424151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.424798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.425069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.425337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.425969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:38.435443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.435556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.435556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.435709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.435709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.435709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.435709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.436037 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:38.436114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.437493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.439648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 231/450] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:38.440464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.451844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 220/451] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:38.451844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.547774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5994e2b-9df6-43e5-8e42-c176926e2233 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.549627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108adfcb-3e07-4350-955f-f6e89ae62656 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.581233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.581850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.582128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.582399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.582942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.583520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.583820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.584099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.592290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.593721 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.594325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.595753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:38.608858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:38.609738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:38.610566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:38.610566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:38.610566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:38.610566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 232/452] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:38.610785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:38.611575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:38.618034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:38.618034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:38.618034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:38.618278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.618352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 221/453] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:38.618352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.654224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.656732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.711292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb5852-a203-4f72-a7ae-2656c32f9d01 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.713686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5d369-9106-4ef7-90a4-e33dba447960 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.743346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.743994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.744272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.744544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.746620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.747232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.747504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.747771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.750990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.752458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.755909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.757329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:38.764460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:38.765307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:38.766138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 233/454] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:38.766597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:38.768450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:38.769501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 222/455] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:38.771196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.869810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5ce47-e677-4f47-9ef3-0bdf9ebc81e4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.870173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeba0b-88cd-4022-91ae-22803a474fb6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:38.871299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.871780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:38.902907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.903519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.903808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.904101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.905095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:38.905688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:38.905962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:38.906230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:38.906858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:38.907199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.908612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.915807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:38.917375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:38.921143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 223/456] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:38.925952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:38.932312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:38.933381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:38.934231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:38.934231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:38.934231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:38.934231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 234/457] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:38.934231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.024778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69938-71e2-4d1c-8655-a3568d9c01b5 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.039827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193b3f9-f3cd-41c1-9a68-b29ada22cdd7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.059817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.060429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.060706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.060976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.061619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:39.061921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.063278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.075394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 224/458] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:39.076331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.077027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.077310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.077584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.078266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:39.078571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.080230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.080561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.092703 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 235/459] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:39.093527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.104584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.110278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.179809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652963b-dd15-490b-afa4-0112413a7011 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.194069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed53b8-ca44-49d0-b912-1ab292724c5c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.212962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.213571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.213852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.214118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.214828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:39.215128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.216586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.227401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.228122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.228408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.228685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.234132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 225/460] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:39.235334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.236670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.236809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:39.248483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:39.249926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 236/461] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:39.251222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.320266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.329449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.338525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65dba2-5b83-4a17-8539-8b8afff6e30d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.353772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0c959-c5ec-4f0c-978a-f27e63a9b848 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.372723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.373329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.373603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.373872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.374498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:39.374797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.376244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.387479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 226/462] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:39.387622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.388207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.388304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.388522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.388748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.398408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.399874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:39.412609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:39.413606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 237/463] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:39.414528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.490773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f15fde-614e-4f7a-855c-3ecd3f9f1391 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.513293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c20b8-0b9b-4516-93c7-4d0599ef23e5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.524909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.525554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.525833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.526107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/6bb4908b26e64c538c2b573f3cf26bf1/roles/7f166188476b48158de248c89d0b9144` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.537295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.541393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=6bb4908b26e64c538c2b573f3cf26bf1, role_id=7f166188476b48158de248c89d0b9144)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.542852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.547983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.548620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.548854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.553047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.553047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.613640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.615847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.616969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 227/464] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:39 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/6bb4908b26e64c538c2b573f3cf26bf1/roles/7f166188476b48158de248c89d0b9144 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:39.617782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.730929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd38ec0-60f7-4cb0-8736-3b37d8c06455 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.747162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 238/465] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:39.747162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.771016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.771637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.771994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.772274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.773003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:39.773312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.774802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.791850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 228/466] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:39.791850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.792782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.843060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:39.872542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e56daa-6861-4e07-9098-7d557e9c7b71 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.895012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb3334-bfe5-43df-b0fb-416071653438 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:39.911158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.912073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.912314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.912696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.920007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.921457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.931005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:39.931609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:39.931909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:39.932182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:39.932904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:39.933209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:39.933300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:39.934093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 10:50:39.939730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:39.940262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:39.940340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 239/467] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:39.940340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:39.958688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 229/468] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:39.963233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.048650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:40.056583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdfa49-8335-4631-91b4-bdecf5779f25 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.070579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9c374-6db4-42e3-ae14-2f17b5ac4033 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.080546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:40.095357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.096291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.096291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.096563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.106302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.107192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.107658 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.108214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.108566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.109445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] RBAC: Adding query filter params (role.id=7f166188476b48158de248c89d0b9144, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=c3d05bdc4c9b4f24b1d58ef157b32ae0) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:40.109754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.110032 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.111163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.117146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 230/469] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/role_assignments?user.id=c3d05bdc4c9b4f24b1d58ef157b32ae0&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=7f166188476b48158de248c89d0b9144 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:40.120743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:40.124831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:40.125781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:40.126600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:40.126600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:40.126600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:40.126600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 240/470] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:40.127257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.255804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29817555-e800-4acb-bf2d-a40bbb9cf0fb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.260733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fda05-a811-4994-8003-cc5eab583244 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.302502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.303413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.303615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.303925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.304662 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:40.304968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.306411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.306799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.307406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.307684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.308003 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.324959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:40.334631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 231/471] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:40.334631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.365340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:40.391776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.393604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.458331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bac236-45ef-41d1-acf6-f7689762c935 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.502238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 241/472] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:40.503217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.503867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.504625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.504986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.505361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.506384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:40.507524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.508978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.525490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 232/473] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:40.526175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.558643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:40.622193 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa881b-afc7-4fed-9be2-267bbb660d85 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.643508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:40.654723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28406a1-954b-479b-8181-c4721bc465ed None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.667930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.668549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.668832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.669287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.692939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.693570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.693846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.694534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.706523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.708193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:40.724224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:40.725129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:40.726683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:40.726683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:40.726683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:40.726683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 233/474] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:40.727184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.757582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.759992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.826818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:40.841937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65661102-ff5c-40e7-ac74-0c4e51e5e9c7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:40.881369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:40.882020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:40.882339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:40.882647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:40.891250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:40.892871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:40.898132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 242/475] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:50:40.898775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:40.912467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:40.914659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 234/476] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:40.916708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:40.920250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:41.027220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ddcfc-3640-46da-884c-b34a79c0a408 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.035430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eef7c-4f9a-4084-b0dd-7981901d0bf3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.079809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.080522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.080808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.081088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.081728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:41.082129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.083238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.084224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.084353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.084628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.084966 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.096298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.097837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.099029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 243/477] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:41.099976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.111030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:41.120119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:41.128825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:41.129793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:41.129793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:41.129793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:41.129793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 235/478] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:41.129793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.201223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:41.218037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c85ca9-5bb1-4d6f-834b-2bcf80dfeea9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.256734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.259063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.259063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.259063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.259063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:41.259411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.261288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.262003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23468d-618a-4762-b382-c9f578c4c724 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.274682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 244/479] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:41.275918 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.306584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.308016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.308354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.308666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.321376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.322950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:41.336379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:41.337631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:41.338472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 236/480] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:41.338926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.357820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:41.402705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aa194-33ae-4551-b55f-1483fba1490c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.447033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.448491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.448940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.449378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.450207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:41.450574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbdc2f-5234-44d6-b4aa-409ab133a727 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.450954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.452683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.466232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:41.472210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 245/481] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:41.472803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.495880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.496488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.496746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.497021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.497872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] RBAC: Adding query filter params (role.id=7f166188476b48158de248c89d0b9144, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=9627e6a9e86e43b694f723bd4109460e) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:41.498180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.499994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.507284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 237/482] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/role_assignments?user.id=9627e6a9e86e43b694f723bd4109460e&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=7f166188476b48158de248c89d0b9144 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:41.508289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.609890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232eaf3-66f6-44ab-a9d4-edbc522c57e1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.623221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43b4f-a5cf-4871-9b42-b6cb1f6068bd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.646595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:41.658068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.659449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.660181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.669713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.669713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:41.669713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.669713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.670014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.670014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.670014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.670014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.673936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.675260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.680471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 246/483] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:41.689310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:41.690191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:41.691025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 238/484] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:41.693341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.698117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:41.778741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:41.847210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc0431-7cc6-4126-ac05-c4a4f4989504 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.869692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7466-90df-4334-8db7-2d9316f85abf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:41.890677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.891306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.891588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.891894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.892636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] RBAC: Adding query filter params (role.id=7f166188476b48158de248c89d0b9144, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=37b474a596cf42efb5e0e5a2a179735e) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:41.892943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.894425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.902103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 247/485] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/role_assignments?user.id=37b474a596cf42efb5e0e5a2a179735e&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=7f166188476b48158de248c89d0b9144 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:41.917346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e951a7f-209f-4c13-91fa-e7f868efd2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:41.917985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e951a7f-209f-4c13-91fa-e7f868efd2cd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:41.918265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e951a7f-209f-4c13-91fa-e7f868efd2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:41.918604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e951a7f-209f-4c13-91fa-e7f868efd2cd None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:41.919330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e951a7f-209f-4c13-91fa-e7f868efd2cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:41.919638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e951a7f-209f-4c13-91fa-e7f868efd2cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:41.921150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e951a7f-209f-4c13-91fa-e7f868efd2cd None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:41.936711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 239/486] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:41.942628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:42.003644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:42.018246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391e22-da74-4b7c-9742-9d37f25423c7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:42.018880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391e22-da74-4b7c-9742-9d37f25423c7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:42.019358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391e22-da74-4b7c-9742-9d37f25423c7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:42.024038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 240/487] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:42.031026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339dac-7df9-4419-9760-1c9db45d81bb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:42.031320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339dac-7df9-4419-9760-1c9db45d81bb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:42.031572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339dac-7df9-4419-9760-1c9db45d81bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:42.074873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bafd21-a147-40ae-93bf-18ca60bd7fe7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:42.076480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-77339dac-7df9-4419-9760-1c9db45d81bb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:42.078056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-77339dac-7df9-4419-9760-1c9db45d81bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:42.117115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:42.117929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:42.118216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:42.118829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:42.119924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] RBAC: Adding query filter params (role.id=7f166188476b48158de248c89d0b9144, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=ce0cb1d976784ea6af783fd193ddac20) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:42.120191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:42.122204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:42.130561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 248/488] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:41 2022] GET /identity/v3/role_assignments?user.id=ce0cb1d976784ea6af783fd193ddac20&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=7f166188476b48158de248c89d0b9144 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:42.178740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 241/489] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:42.469713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:42.521710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e734a-7327-4f4c-a1a3-3da357525ca3 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:42.522235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e734a-7327-4f4c-a1a3-3da357525ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:42.522235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e734a-7327-4f4c-a1a3-3da357525ca3 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:42.523901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 242/490] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:42.528910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a6804-081f-4af5-af8d-fe83ab76b557 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:42.529197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a6804-081f-4af5-af8d-fe83ab76b557 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:42.529473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a6804-081f-4af5-af8d-fe83ab76b557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:42.551737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b45a6804-081f-4af5-af8d-fe83ab76b557 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:42.553170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b45a6804-081f-4af5-af8d-fe83ab76b557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:42.579509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:42.581504 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50725-d654-46a3-bab2-58fbc59bc888 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:42.603447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:42.627677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 243/491] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:50:42.629440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:42.630098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:42.630500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:42.630777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:42.646616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:42.652237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:42.653099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:42.653907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:42.658222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 249/492] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:42.669265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a6804-081f-4af5-af8d-fe83ab76b557 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:42.806982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a6804-081f-4af5-af8d-fe83ab76b557 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:42.843147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:42.843755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:42.844061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:42.844333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:42.845001 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:42.845364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:42.846742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:42.855582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 244/493] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:42.861125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:42.993622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6926ae3-0b08-4020-a001-1dce53d7e0a6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.024886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.033857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.033857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.033857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.033857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:43.045996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:43.047453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:43.052302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:43.079243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:43.080208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:43.081050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:43.081050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:43.081050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:43.081050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 250/494] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:43.086444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.133182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a69c65-e337-400a-903c-b195ce2e2cb6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.135746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:43.168843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.169476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.169756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.170030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:43.181325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:43.186679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:43.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:43.188923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 245/495] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:43.200950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.216583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386a805-2d7c-490c-8bee-2a3d922e70a2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.280549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.281580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.282068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.282551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:43.283400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:43.283933 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:43.285580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:43.299853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:43.307683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 251/496] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:43.316888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.319475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23327a9-0334-4011-ae7f-96096a7b5fab None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.356750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.357366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.357704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.357981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:43.358803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:43.359217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:43.360709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:43.369990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 246/497] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:43.376207 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.442062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:43.483824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e06859-5e35-4b41-b981-27b640611813 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.523280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.523951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.524236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.524512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:43.537926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:43.539373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:43.564213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:43.565340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:43.566451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 252/498] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:43.571082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.601631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25ad4c-ffa0-4408-aa4c-eed86fa2fe1b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.662338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.664943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.665228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.665498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:43.687924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:43.689411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:43.694139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:43.722750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:43.724313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:43.725666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:43.725666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:43.725666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:43.729717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 247/499] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:43.732824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.771415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8676a-eee3-4b6c-a358-82c63c5fd28c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.840013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.840627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.840904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.841175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:43.841904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:43.842209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:43.843622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:43.880416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 253/500] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:43.886108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:43.892192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:43.927989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c834b4-9497-475f-8bc1-f8033d4dd39b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:43.998433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:43.999046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:43.999327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:43.999602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.000273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:44.000579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.001965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.027068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 248/501] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:44.032855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:44.054975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01e7cc-56a1-4290-8d33-b76de83b35dd None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:44.091418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:44.092560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.092847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.093127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.167051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.168740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.172103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:44.213603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:44.298547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 254/502] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 26 10:50:44.305132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3fd8a-1144-43e1-8569-ee03791bdbb9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:44.307398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:44.374879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9e942-352d-4627-a126-c28c7f1ef3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:44.376064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9e942-352d-4627-a126-c28c7f1ef3eb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.376650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9e942-352d-4627-a126-c28c7f1ef3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.377165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9e942-352d-4627-a126-c28c7f1ef3eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.386099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9e942-352d-4627-a126-c28c7f1ef3eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.387957 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9e942-352d-4627-a126-c28c7f1ef3eb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6ff9e942-352d-4627-a126-c28c7f1ef3eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:44.401906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:44.402911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:44.403858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:44.410959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 249/503] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:44.412770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64272a3-e100-45a0-ba05-14dbde03b952 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.413773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64272a3-e100-45a0-ba05-14dbde03b952 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.414242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64272a3-e100-45a0-ba05-14dbde03b952 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.416108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 250/504] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:44.417165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64272a3-e100-45a0-ba05-14dbde03b952 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:44.435363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b1905-ac09-4635-8fc7-1f75c699ec73 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:44.437752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:44.471852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb353cf-eaca-4f56-af6d-45aa580d1470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:44.472413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb353cf-eaca-4f56-af6d-45aa580d1470 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.472685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb353cf-eaca-4f56-af6d-45aa580d1470 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.473548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb353cf-eaca-4f56-af6d-45aa580d1470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.484364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb353cf-eaca-4f56-af6d-45aa580d1470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.485899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb353cf-eaca-4f56-af6d-45aa580d1470 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-6bb353cf-eaca-4f56-af6d-45aa580d1470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:44.503663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:44.505196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:44.506065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:44.506065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:44.506065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:44.506065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 255/505] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:44.506972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d562-262d-4a10-8b11-ed9c659f8c68 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.506972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d562-262d-4a10-8b11-ed9c659f8c68 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.507163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d562-262d-4a10-8b11-ed9c659f8c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.531027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1d86d562-262d-4a10-8b11-ed9c659f8c68 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:44.532304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1d86d562-262d-4a10-8b11-ed9c659f8c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:44.540833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64272a3-e100-45a0-ba05-14dbde03b952 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:44.555767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:44.577374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:44.578280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.578777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.579230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.580176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:44.580663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.582323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.602980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 251/506] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:44.605820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:44.614252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 256/507] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:44.615211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86d562-262d-4a10-8b11-ed9c659f8c68 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:44.732450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa263d-6bf2-4456-ad53-99a5add3dd7b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:44.771052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:44.773083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ff69f-347b-4401-b359-f5d629ef9ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:44.773700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff69f-347b-4401-b359-f5d629ef9ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.773979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff69f-347b-4401-b359-f5d629ef9ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.774319 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff69f-347b-4401-b359-f5d629ef9ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.774977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ff69f-347b-4401-b359-f5d629ef9ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:44.775281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ff69f-347b-4401-b359-f5d629ef9ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.776690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ff69f-347b-4401-b359-f5d629ef9ef0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.786808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86d562-262d-4a10-8b11-ed9c659f8c68 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:44.788989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 252/508] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:44.790243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e5a1-0e79-488b-9cf2-591cd49d7d46 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.790584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e5a1-0e79-488b-9cf2-591cd49d7d46 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.790862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e5a1-0e79-488b-9cf2-591cd49d7d46 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.792362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 253/509] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:44.795248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e5a1-0e79-488b-9cf2-591cd49d7d46 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:44.830152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946fdee-96e6-4739-9a45-79c45c750395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:44.830762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:44.831892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946fdee-96e6-4739-9a45-79c45c750395 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.832167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946fdee-96e6-4739-9a45-79c45c750395 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.833720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946fdee-96e6-4739-9a45-79c45c750395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.844590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946fdee-96e6-4739-9a45-79c45c750395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.846082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946fdee-96e6-4739-9a45-79c45c750395 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d946fdee-96e6-4739-9a45-79c45c750395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:44.864991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:44.865939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:44.866833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:44.866833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:44.866833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:44.866833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 257/510] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:44.867448 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e8b3-d1b6-4087-b471-f35154fd47eb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.867559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e8b3-d1b6-4087-b471-f35154fd47eb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.867751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e8b3-d1b6-4087-b471-f35154fd47eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.888133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1501e8b3-d1b6-4087-b471-f35154fd47eb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:44.889412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1501e8b3-d1b6-4087-b471-f35154fd47eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:44.916158 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e5a1-0e79-488b-9cf2-591cd49d7d46 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:44.949887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:44.950518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.950799 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.951071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.958571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:44.959963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:44.975814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 258/511] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:44.975814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b350-9b68-42db-9423-ae7130f565c5 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:44.975814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b350-9b68-42db-9423-ae7130f565c5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:44.975814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b350-9b68-42db-9423-ae7130f565c5 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:44.975814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 259/512] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:44.975814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b350-9b68-42db-9423-ae7130f565c5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:44.976307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:44.977100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 254/513] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:44.977971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.057898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.073797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.083228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7a14c-5e9c-4785-a0d5-490b10a832e7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.094656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b350-9b68-42db-9423-ae7130f565c5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.119219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.119875 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.120160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.120434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.130649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.132252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.132908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978600c-0591-4706-982a-545bef17f217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.133572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978600c-0591-4706-982a-545bef17f217 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.133855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978600c-0591-4706-982a-545bef17f217 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.134257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978600c-0591-4706-982a-545bef17f217 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.134767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978600c-0591-4706-982a-545bef17f217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:45.135485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978600c-0591-4706-982a-545bef17f217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.136710 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978600c-0591-4706-982a-545bef17f217 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.153411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 260/514] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:45.153587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:45.154467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:45.155339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:45.155339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:45.155339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:45.155521 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043cd83-3f51-4cff-948c-5cce424288be None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.155589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 255/515] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:45.155589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.155708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043cd83-3f51-4cff-948c-5cce424288be None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.156495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043cd83-3f51-4cff-948c-5cce424288be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.178255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c043cd83-3f51-4cff-948c-5cce424288be None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:45.180305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c043cd83-3f51-4cff-948c-5cce424288be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:45.259835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 261/516] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:45.259835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043cd83-3f51-4cff-948c-5cce424288be None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.260889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79295e85-bf5c-4223-bf3b-d68e75c6c60a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.303818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.304359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.304645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.304917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.305603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:45.305937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.307329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.319154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.320121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 256/517] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:45.320931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.325463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.368985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043cd83-3f51-4cff-948c-5cce424288be None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.405198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.406142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.406420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.406687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.413937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.415438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:45.427258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:45.428183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:45.429261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:45.432867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 262/518] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:45.433548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a803-8b3d-42d6-8d5d-e9a44e8b7f11 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.433669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.470136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.470744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.471024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.471294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.472197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:45.472305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.474216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.491116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 257/519] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:45.491116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.543197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930cc80-d124-4e8f-b79a-afb158115711 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.556102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.560973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.578103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.578727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.579060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.579336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.579941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.581347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:45.591737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:45.592775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:45.593635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 263/520] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:45.594416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab60cc-5dea-4f08-abc5-cfe9b728dd41 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.594498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.628300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.628921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.629433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.629709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.630425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] RBAC: Adding query filter params (role.id=7f166188476b48158de248c89d0b9144, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=6bb4908b26e64c538c2b573f3cf26bf1) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:45.630733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.632191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.638085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 258/521] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/role_assignments?user.id=6bb4908b26e64c538c2b573f3cf26bf1&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=7f166188476b48158de248c89d0b9144 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:45.638838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.717476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9444926-5942-4abf-951f-5121b2f810ce None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.756524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.757041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8d281-c0a2-42e5-ac2a-e5f11e27c3ef None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.757151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.757413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.757683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.758379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:45.758688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.760447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.774234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 264/522] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:45.775189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.790460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.798927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.799551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.806706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.806706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.806706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.806706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.825258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:45.882134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8576a-f3dd-4881-8d93-f39921d16ccd None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:45.891915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 259/523] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:50:45 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:45.893023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:45.945660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:45.946700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:45.947915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:45.948241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] PATH_INFO: `/v3/services/network` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:45.948886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:45.950365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:45.969396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:45.970451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:45.971275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 265/524] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:45.972079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:46.052847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c904bf-ef28-41b6-a504-0631675da9e3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:46.097622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:46.099825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:46.100316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:46.101056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:46.101944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:46.102458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:46.102632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01967f-c303-4edd-bbdf-dac53c1f4f4e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:46.104411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:46.110143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 260/525] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 26 10:50:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:46.111664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:46.115995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:46.151195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:46.152010 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:46.152300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:46.152607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/c3d05bdc4c9b4f24b1d58ef157b32ae0/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:46.155283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=c3d05bdc4c9b4f24b1d58ef157b32ae0, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:46.156794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:46.172701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:46.226174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3419e8-b04b-42b9-814b-3203b424aefa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:46.243127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 266/526] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:45 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/c3d05bdc4c9b4f24b1d58ef157b32ae0/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:46.247933 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:46.285182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:46.286327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:46.286837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:46.287311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:46.288175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] RBAC: Adding query filter params (name=network) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:46.288682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:46.290371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:46.301777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 261/527] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:50:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:46.307547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:46.373289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f690a-5b61-4123-8ec6-d8c7496f6857 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:46.412446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd0671-a317-429b-861a-db13bb6d92f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:46.413066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd0671-a317-429b-861a-db13bb6d92f8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:46.413340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd0671-a317-429b-861a-db13bb6d92f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:46.413608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd0671-a317-429b-861a-db13bb6d92f8 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:46.414359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd0671-a317-429b-861a-db13bb6d92f8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:46.414658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd0671-a317-429b-861a-db13bb6d92f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:46.416137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd0671-a317-429b-861a-db13bb6d92f8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:46.422358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 267/528] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 26 10:50:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:46.503435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28600f6-fce5-41d5-827e-d44d85accf3e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:46.520366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:46.557981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361d774-2be4-4ae3-b8c2-e54098f30bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:46.559041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361d774-2be4-4ae3-b8c2-e54098f30bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:46.559837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361d774-2be4-4ae3-b8c2-e54098f30bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:46.560125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361d774-2be4-4ae3-b8c2-e54098f30bf1 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:46.560787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361d774-2be4-4ae3-b8c2-e54098f30bf1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:46.561097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361d774-2be4-4ae3-b8c2-e54098f30bf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:46.562519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361d774-2be4-4ae3-b8c2-e54098f30bf1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:46.569973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 262/529] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:50:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:48.213268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1c19-413a-4196-8381-0d9fa6c2e8c8 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:48.214269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1c19-413a-4196-8381-0d9fa6c2e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:48.214747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1c19-413a-4196-8381-0d9fa6c2e8c8 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:48.216548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 268/530] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:48.221999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4572f73-3e2b-4daa-aacd-5446d8d4cb18 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:48.222396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4572f73-3e2b-4daa-aacd-5446d8d4cb18 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:48.222608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4572f73-3e2b-4daa-aacd-5446d8d4cb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:48.248871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b4572f73-3e2b-4daa-aacd-5446d8d4cb18 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:48.250085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b4572f73-3e2b-4daa-aacd-5446d8d4cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:48.373415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 263/531] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:48.451006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0207fbc-1599-47bb-af1c-3987cf25365f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:48.451336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0207fbc-1599-47bb-af1c-3987cf25365f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:48.451615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0207fbc-1599-47bb-af1c-3987cf25365f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:48.452986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 269/532] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:48.458518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499585-3850-427f-9887-1938dcd310da None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:48.458811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499585-3850-427f-9887-1938dcd310da None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:48.459095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499585-3850-427f-9887-1938dcd310da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:48.484828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e9499585-3850-427f-9887-1938dcd310da None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:48.486040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e9499585-3850-427f-9887-1938dcd310da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:48.573176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 264/533] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:48.698622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0207fbc-1599-47bb-af1c-3987cf25365f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:48.743653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:48.819454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0207fbc-1599-47bb-af1c-3987cf25365f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:48.855573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:48.856238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:48.856510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:48.856774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:48.868590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:48.873774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:48.874808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:48.876333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 270/534] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:48.882001 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9499585-3850-427f-9887-1938dcd310da None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:48.920174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:48.995424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9499585-3850-427f-9887-1938dcd310da None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:48.999349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.085018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be6248-95c0-4492-a4d7-f73cf66e113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.085625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6248-95c0-4492-a4d7-f73cf66e113a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.085902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6248-95c0-4492-a4d7-f73cf66e113a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.086169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6248-95c0-4492-a4d7-f73cf66e113a None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.086825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be6248-95c0-4492-a4d7-f73cf66e113a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:49.087121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be6248-95c0-4492-a4d7-f73cf66e113a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:49.088517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be6248-95c0-4492-a4d7-f73cf66e113a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:49.105831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 265/535] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:49.107336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8864c6-8a2d-4fae-9098-252e8ee1516e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.107824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8864c6-8a2d-4fae-9098-252e8ee1516e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.108257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8864c6-8a2d-4fae-9098-252e8ee1516e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.109883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 266/536] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:49.115605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8864c6-8a2d-4fae-9098-252e8ee1516e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.154736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:49.165580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425730-15df-4ed7-a034-24bc1fa6bb8a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.209390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710b8d5-a8fe-4831-b053-dac241ddb542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.210301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710b8d5-a8fe-4831-b053-dac241ddb542 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.210954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710b8d5-a8fe-4831-b053-dac241ddb542 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.211763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710b8d5-a8fe-4831-b053-dac241ddb542 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.224416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:49.228551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710b8d5-a8fe-4831-b053-dac241ddb542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:49.229118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8864c6-8a2d-4fae-9098-252e8ee1516e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-5710b8d5-a8fe-4831-b053-dac241ddb542 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:49.236523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:49.239677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:49.240725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:49.243821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 271/537] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:49.245532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf5f28-5418-4468-97ee-c40acaf1cc70 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.245966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf5f28-5418-4468-97ee-c40acaf1cc70 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.246362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf5f28-5418-4468-97ee-c40acaf1cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.272087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-08bf5f28-5418-4468-97ee-c40acaf1cc70 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:49.274700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-08bf5f28-5418-4468-97ee-c40acaf1cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:49.279001 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.279817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.280217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.280589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.292827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:49.294963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:49.312278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:49.313494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:49.314615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:49.314615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:49.314615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:49.315932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 267/538] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:49.321399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.375202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 272/539] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:49.376005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf5f28-5418-4468-97ee-c40acaf1cc70 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.433505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32022300-afa8-44fd-a624-e8bedcf3a274 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.470210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa7621-fd1c-438e-8cbe-a2aaf80a5931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.470921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:49.471279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa7621-fd1c-438e-8cbe-a2aaf80a5931 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.471565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa7621-fd1c-438e-8cbe-a2aaf80a5931 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.472341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa7621-fd1c-438e-8cbe-a2aaf80a5931 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.473299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa7621-fd1c-438e-8cbe-a2aaf80a5931 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:49.473660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa7621-fd1c-438e-8cbe-a2aaf80a5931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:49.475499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa7621-fd1c-438e-8cbe-a2aaf80a5931 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:49.484733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 268/540] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:49.485934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ed9d-514e-416e-ad06-c2dc95fed961 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.493171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ed9d-514e-416e-ad06-c2dc95fed961 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.493171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469ed9d-514e-416e-ad06-c2dc95fed961 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.493171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 269/541] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:49.493171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a98d53-558f-45ac-b7f4-8c09d674d816 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.493171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a98d53-558f-45ac-b7f4-8c09d674d816 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.493171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a98d53-558f-45ac-b7f4-8c09d674d816 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.493171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 270/542] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:49.493668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a98d53-558f-45ac-b7f4-8c09d674d816 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.500875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf5f28-5418-4468-97ee-c40acaf1cc70 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.508257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:49.542168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f008b1-cc8b-47fe-ac39-6a12e6b5b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.542787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f008b1-cc8b-47fe-ac39-6a12e6b5b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.543065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f008b1-cc8b-47fe-ac39-6a12e6b5b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.543326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f008b1-cc8b-47fe-ac39-6a12e6b5b9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.543993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f008b1-cc8b-47fe-ac39-6a12e6b5b9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:49.544298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f008b1-cc8b-47fe-ac39-6a12e6b5b9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:49.546162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f008b1-cc8b-47fe-ac39-6a12e6b5b9e2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:49.559744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 273/543] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:49.561033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88528d03-223d-405a-af31-a1589f130704 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.561321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88528d03-223d-405a-af31-a1589f130704 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.561595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88528d03-223d-405a-af31-a1589f130704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.580934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-88528d03-223d-405a-af31-a1589f130704 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:49.582583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-88528d03-223d-405a-af31-a1589f130704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:49.605025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a98d53-558f-45ac-b7f4-8c09d674d816 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.639874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.640495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.640769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.641036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.651450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:49.652953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:49.656634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 274/544] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:49.657917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603e08-68b5-46a4-af0a-6df92fd4e74f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.658202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603e08-68b5-46a4-af0a-6df92fd4e74f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.658475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603e08-68b5-46a4-af0a-6df92fd4e74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.676867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1a603e08-68b5-46a4-af0a-6df92fd4e74f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:49.678039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1a603e08-68b5-46a4-af0a-6df92fd4e74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:49.678373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:49.679335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:49.680273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:49.680273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:49.680273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:49.680273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 271/545] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:49.681043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.745108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:49.759879 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 275/546] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:50:49.759879 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a603e08-68b5-46a4-af0a-6df92fd4e74f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.761015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:49.795819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a185e-97be-4307-b36c-7415856da779 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.839237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.839986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.840334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.840670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.852208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:49.854055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:49.866236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:49.867198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 272/547] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:49.868782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.878066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a603e08-68b5-46a4-af0a-6df92fd4e74f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.917579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:49.918573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:49.919416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:49.920186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:49.932513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:49.938387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:49.939287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:49.940138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:49.944405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 276/548] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:49.945461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:49.972982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab735ac-e9d5-4764-b0d3-911ad3bb370b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:49.995429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:50.008293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.008986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.009233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.009479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.010438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:50.010735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.012123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.014828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:50.028987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 273/549] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:50.028987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.063843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a43db-5b7a-4058-8043-ea0a5dec3b72 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.097595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.098199 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.098465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.098735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.099452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:50.099752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.101195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.118278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 277/550] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 26 10:50:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:50.119080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.134449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2d55f-c53d-414f-a83b-f7824e215efc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.167118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.167712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.168019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.168287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.168921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.170528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.217973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0c521-c3ef-4ba4-a3c7-4ae0822316ab None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.234558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:50.251450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.252073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.252483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.252678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.253316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:50.253614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.255679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.272672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 274/551] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:50:50 2022] POST /identity/v3/services => generated 240 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:50.272672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.273265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 278/552] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:50.273265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.322326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:50.393587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7fcad-90eb-406d-86ad-b32e4a7ff7a8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.401820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938bc29-dd8a-4c4b-ac36-ddf32a93208f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.437162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.438029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.438365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.439019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.450350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.452750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.456868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.457607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.457889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.458375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.458806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.460280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:50.467336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:50.468518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 275/553] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:50:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:50.469465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.536272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 279/554] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:50:50 2022] POST /identity/v3/services => generated 251 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:50.537144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.598852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:50.602830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f52db7-b558-475a-ac37-bbe4cbd2a504 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.656586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.657649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.658151 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.659343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.662677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:50.675574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.677735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:50.698638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:50.699684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:50.700619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:50.700619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:50.700619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:50.702922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 276/555] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:50.704832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.797340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb96c-f04c-4abe-a69b-09144734ff49 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.846842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ec409-196b-4838-8145-0f5047447190 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:50.853283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.854221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.856170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.856170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.871126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.872635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.887728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:50.888524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:50.888803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:50.889081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:50.889493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:50.890382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:50.891279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:50.891279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:50.891279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:50.891455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:50.891455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:50.891577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 280/556] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:50.891577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.891702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:50.917066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 277/557] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 26 10:50:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:50.922317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:50.932096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.008332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.018994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dfbce-21c6-435a-b11f-0be6c9e1c852 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.034765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720755-4e75-455a-b7b0-536d9fb76565 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.071449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.072943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.073374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.074076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.074903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:51.075522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.077184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.081333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.082478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.082757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.083032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.083679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:51.084008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.085382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.093233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 281/558] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:51.094822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.106784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 278/559] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:51.106784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.189328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.225124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54023b1f-c832-4e6f-986c-c278d793505c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.232388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e287-fb3e-4fae-b633-788dc1767187 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.262073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.262720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.262981 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.263254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.270117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.275648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.275648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.275648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.276399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.276399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.279632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.281403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:51.291909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:51.293065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:51.294148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:51.294148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:51.294148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:51.296358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:51.297387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:51.298666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:51.299209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 282/560] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:51.299209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.300332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 279/561] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:51.301516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.302541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.403359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d42384-4205-41fd-b81d-b30dc99c0de2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.409477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de289323-1136-4d8b-96aa-138dad4c551d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.438987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.439623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.440002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.440222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.448239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.449124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.449698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.449976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.450380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.451225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:51.451440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.455965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.457267 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.458965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:51.463277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:51.464261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:50:51.465210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:51.469282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 283/562] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:51.470130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.477306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 280/563] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:51.478383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.540347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.579643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621614e4-0ec6-4b13-82f4-fd3db2e56aaa None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.590229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6900f-1f89-4adc-9279-68fc5d25d8d8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.616852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.617497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.617777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.618047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.618766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:51.619073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.620621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.631865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.632817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.633297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.633767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.634169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 284/564] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:51.635195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:51.635526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.635936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.637554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.656633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 281/565] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:51.657696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.704333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.749789 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16617a2e-1219-426c-8577-38a2b978b900 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.761718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89dbb4-fdf5-4951-8919-ef0c19268dad None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:51.786707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.787273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.787560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.787864 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.795266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.797048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.798899 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:51.799775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:51.800318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:51.801432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/6bb4908b26e64c538c2b573f3cf26bf1/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:51.803602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=6bb4908b26e64c538c2b573f3cf26bf1, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:51.805266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:51.809701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:51.810613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:51.811583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:51.812467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:51.815827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 285/566] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:51.816737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.875305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 282/567] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:51 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/6bb4908b26e64c538c2b573f3cf26bf1/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:50:51.876403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:51.957563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed45b-887e-4131-89e4-a6dadff0000f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:52.042418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:52.043098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:52.043374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:52.043641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:52.107771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6817a3a-e09c-42d8-a29e-217cd0af7baf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:52.111001 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:52.142877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:52.155218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:52.163520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:52.163898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:52.164881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:52.165507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:52.165994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:52.167160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:52.168657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:52.170408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:52.195815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 283/568] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:52.198374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:52.377118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 286/569] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:50:52.383468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:52.407910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fded5-6098-49b8-a254-fc7081c144f8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:52.493358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adf219-e002-4df3-a9d9-c06f4ea5b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:52.494033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adf219-e002-4df3-a9d9-c06f4ea5b8a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:52.494363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adf219-e002-4df3-a9d9-c06f4ea5b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:52.494584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adf219-e002-4df3-a9d9-c06f4ea5b8a1 None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/d64e1522d1d4425bbb88d78e4f8863d6/roles/7f166188476b48158de248c89d0b9144` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:52.532947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:52.549285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adf219-e002-4df3-a9d9-c06f4ea5b8a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=d64e1522d1d4425bbb88d78e4f8863d6, role_id=7f166188476b48158de248c89d0b9144)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:52.551505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adf219-e002-4df3-a9d9-c06f4ea5b8a1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:52.555463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bec19-d4cf-4d7b-a6a1-04255ff7f92b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:52.597373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308f148-b058-4e13-97bf-dc176f3b6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:52.602815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308f148-b058-4e13-97bf-dc176f3b6164 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:52.602815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308f148-b058-4e13-97bf-dc176f3b6164 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:52.602815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308f148-b058-4e13-97bf-dc176f3b6164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:52.616942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308f148-b058-4e13-97bf-dc176f3b6164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:52.618388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308f148-b058-4e13-97bf-dc176f3b6164 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-6308f148-b058-4e13-97bf-dc176f3b6164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:52.636788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:52.637834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:52.639409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:52.639409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:52.639409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:52.639409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 287/570] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:52.639409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfddcb7-aae9-4a7f-981b-3149fdae4441 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:52.639703 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfddcb7-aae9-4a7f-981b-3149fdae4441 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:52.639952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfddcb7-aae9-4a7f-981b-3149fdae4441 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:52.642532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 288/571] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:52.646693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 284/572] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:50:52 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/d64e1522d1d4425bbb88d78e4f8863d6/roles/7f166188476b48158de248c89d0b9144 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:50:52.650042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfddcb7-aae9-4a7f-981b-3149fdae4441 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:52.650779 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22534bb7-1357-4490-a453-c2a0297bcfbb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:52.652476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22534bb7-1357-4490-a453-c2a0297bcfbb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:52.652955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22534bb7-1357-4490-a453-c2a0297bcfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:52.706358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-22534bb7-1357-4490-a453-c2a0297bcfbb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:52.710495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-22534bb7-1357-4490-a453-c2a0297bcfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:52.743433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:52.881944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 285/573] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:50:52.908776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfddcb7-aae9-4a7f-981b-3149fdae4441 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:52.948743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b67203-122c-4406-9dd2-62563a9b4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:52.949542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67203-122c-4406-9dd2-62563a9b4602 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:52.949829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67203-122c-4406-9dd2-62563a9b4602 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:52.950172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67203-122c-4406-9dd2-62563a9b4602 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:52.950884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b67203-122c-4406-9dd2-62563a9b4602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:52.951190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b67203-122c-4406-9dd2-62563a9b4602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:52.952963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b67203-122c-4406-9dd2-62563a9b4602 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:52.956147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:52.970209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 289/574] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:52.978711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22534bb7-1357-4490-a453-c2a0297bcfbb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.111913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618498c5-50c7-4910-8f8e-25335f7d310c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.112248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618498c5-50c7-4910-8f8e-25335f7d310c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.112477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618498c5-50c7-4910-8f8e-25335f7d310c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.114896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 290/575] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:53.121238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d3ea5-430a-4336-a571-5fee78634cf3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.121530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d3ea5-430a-4336-a571-5fee78634cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.121779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d3ea5-430a-4336-a571-5fee78634cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.137502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22534bb7-1357-4490-a453-c2a0297bcfbb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:53.157890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-290d3ea5-430a-4336-a571-5fee78634cf3 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:53.157890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-290d3ea5-430a-4336-a571-5fee78634cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:53.184769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:53.185775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.186259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.186742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.194595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:53.196461 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:53.220011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:53.220958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:53.224041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 286/576] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:53.231265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.240538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:53.280811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 291/577] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:50:53.281867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d3ea5-430a-4336-a571-5fee78634cf3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.370158 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0146bb9-d93e-477f-acfe-f468cda3d463 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:53.400394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d3ea5-430a-4336-a571-5fee78634cf3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:53.407743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:53.408786 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.409677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.410722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.411741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:53.412278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:53.413975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:53.427076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 287/578] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:53.434280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.436306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:53.453934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:53.455361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.456064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.457891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] PATH_INFO: `/v3/services/network` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.458684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:53.461281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:53.480817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:53.481932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:53.482806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:53.486625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 292/579] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:53.492993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.551581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:53.592454 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7fccc-6276-4e38-b445-4702d46041c3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:53.631043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:53.632108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.632636 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.633409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.634347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=c3d05bdc4c9b4f24b1d58ef157b32ae0) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:53.635133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:53.637255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:53.647981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 288/580] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/role_assignments?user.id=c3d05bdc4c9b4f24b1d58ef157b32ae0&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:53.712665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9275baf-3591-4636-9491-6e5aeb7f349c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:53.726457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.757281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:53.757907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.758188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.758464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.759108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] RBAC: Adding query filter params (name=network) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:53.759417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:53.761125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:53.769880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 293/581] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:53.776901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.862122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:53.911144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a283d8-9216-4c00-8a83-b5bcf7c439bc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:53.917461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd1015-790e-4fdc-be5f-5504ac54e053 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:53.947036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:53.960660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:53.960852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.961004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.961282 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.961924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:53.963825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:53.966189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:53.967011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:53.967476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:53.967959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:53.968761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] RBAC: Adding query filter params (type=network) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:53.969307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:53.970866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:53.975547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:53.976683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:53.977945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 289/582] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:53.979910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 294/583] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:53.982448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:53.985886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:54.129649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719984-27c0-4237-b0ad-3373ea3430cc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:54.145748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242d483-ce1e-4644-b1f9-568c0ecf3f4f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:54.180625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:54.202458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:54.203341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.205130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.205579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.206364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:54.208025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:54.209579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:54.217181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:54.218241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.219818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.220577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 290/584] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:54.221175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.222134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] RBAC: Adding query filter params (interface=public, service_id=ec1041e822914cafa2d8bfca2a2a9e28, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:54.223318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:54.224993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:54.226694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:54.250079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 295/585] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:50:53 2022] GET /identity/v3/endpoints?service_id=ec1041e822914cafa2d8bfca2a2a9e28&interface=public®ion_id=RegionOne => generated 194 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:54.254473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:54.343302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:54.439199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800e857-a1dc-4ed9-a2a2-2a6a68f4ec99 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:54.459308 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff66b4-8799-4820-ae6b-6b6df785117d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:54.507312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:54.507853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.508127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.508387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.509009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:54.509485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:54.511136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:54.522942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 291/586] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 26 10:50:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:54.525850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c8413-7064-4628-a2de-92291b50a142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:54.526460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8413-7064-4628-a2de-92291b50a142 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.526815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8413-7064-4628-a2de-92291b50a142 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.527028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8413-7064-4628-a2de-92291b50a142 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.528096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:54.528855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c8413-7064-4628-a2de-92291b50a142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:54.530214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c8413-7064-4628-a2de-92291b50a142 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:54.543046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 296/587] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:50:54 2022] GET /identity/v3/services => generated 772 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:54.545735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bdf5e-7247-4906-8d17-da901da51783 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.546030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bdf5e-7247-4906-8d17-da901da51783 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.546312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bdf5e-7247-4906-8d17-da901da51783 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.551833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 297/588] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:54.555207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453894-3d01-4356-9ba6-cc7ff8cde83a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.555817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453894-3d01-4356-9ba6-cc7ff8cde83a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.556320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453894-3d01-4356-9ba6-cc7ff8cde83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.576429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d8453894-3d01-4356-9ba6-cc7ff8cde83a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:54.578084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d8453894-3d01-4356-9ba6-cc7ff8cde83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:54.628237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:54.669474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 298/589] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:50:54.721187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092984b-fcb1-4d8d-a1c8-01288e6ccbef None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:54.745748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:54.761353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:54.761890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.762163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.762430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.763133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] RBAC: Adding query filter params (interface=public, service_id=7104b1a76a71465c89ea2b2c775c379f, region_id=RegionOne) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:54.763434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:54.764855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:54.774337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 292/590] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:50:54 2022] GET /identity/v3/endpoints?service_id=7104b1a76a71465c89ea2b2c775c379f&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:54.780810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8453894-3d01-4356-9ba6-cc7ff8cde83a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:54.888558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8453894-3d01-4356-9ba6-cc7ff8cde83a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:54.924628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:54.925238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:54.925548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:54.925817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:54.926418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:54.927803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:54.933566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 299/591] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:50:54 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:54.958872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:55.111032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076cc43-5a6a-4db7-a37e-1daabbd9242d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:55.158992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:55.159650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:55.160466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:55.160741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:55.181390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:55.186783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:55.187706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:55.188997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 293/592] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:55.194790 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:55.239852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:55.301203 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422abac-6ebd-43fc-9b29-d6727ae6ed05 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:55.345519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1fa1b-8c41-443e-b548-96719a38b321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:55.346145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1fa1b-8c41-443e-b548-96719a38b321 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:55.346423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1fa1b-8c41-443e-b548-96719a38b321 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:55.346695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1fa1b-8c41-443e-b548-96719a38b321 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:55.347459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1fa1b-8c41-443e-b548-96719a38b321 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:55.347768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1fa1b-8c41-443e-b548-96719a38b321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:55.349301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1fa1b-8c41-443e-b548-96719a38b321 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:55.360364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 300/593] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:50:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:55.366674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:55.424747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:55.468472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe22b4-1cbc-4b28-bb39-17fbe551ac6d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:55.468995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe22b4-1cbc-4b28-bb39-17fbe551ac6d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:55.468995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe22b4-1cbc-4b28-bb39-17fbe551ac6d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:55.470471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 301/594] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:50:55.481097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7a5e-ef1e-4c54-827b-d98e314ba553 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:55.481397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7a5e-ef1e-4c54-827b-d98e314ba553 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:55.481642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7a5e-ef1e-4c54-827b-d98e314ba553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:55.499850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5d61-be99-4e38-a0c1-ad5457d745da None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:55.517612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-dceb7a5e-ef1e-4c54-827b-d98e314ba553 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:55.518979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dceb7a5e-ef1e-4c54-827b-d98e314ba553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:55.554783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:55.555346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:55.555625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:55.555934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:55.573357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:55.574841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:55.606715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:55.607732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:55.608669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:55.608669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:55.608669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:55.608669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 294/595] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:55.616476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:55.646554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 302/596] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:50:55.728815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:55.735043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c65af9-7433-40c5-aed1-efd1f307473f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:55.771007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:55.771630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:55.771939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:55.772217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:55.772862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:55.773162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:55.774557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:55.787562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 295/597] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:55.792705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7a5e-ef1e-4c54-827b-d98e314ba553 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:55.835954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:55.924831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7a5e-ef1e-4c54-827b-d98e314ba553 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:55.969309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:55.969949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:55.970228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:55.970501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:55.981454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:55.982903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:55.990810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:56.011425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:56.012484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 10:50:56.013500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:56.016251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 303/598] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:50:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:56.017979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:56.162558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f430c2-d534-46c8-bb21-937fc8648edf None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:56.184151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:56.199767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d97a1-ffea-4ce7-83d3-0572f2843c93 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:56.226674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:56.227317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.227594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.227898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.249594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:50:56.255114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:56.256167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:50:56.257036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:56.258198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 296/599] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:50:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:50:56.263597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:56.270512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:56.271140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.271419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.271718 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.272498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:56.272807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:56.274310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:56.306547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:56.309471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 304/600] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 26 10:50:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:56.315290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:56.447370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c253-d2fc-40d6-a6a4-49cba0b0fba8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:56.464949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfc44e-c5c6-4d29-8e35-e3b8d52071fb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:56.488916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a3637-d5bb-491f-858f-e06b2a380a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:56.489938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3637-d5bb-491f-858f-e06b2a380a55 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.490278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3637-d5bb-491f-858f-e06b2a380a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.490601 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3637-d5bb-491f-858f-e06b2a380a55 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.491544 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a3637-d5bb-491f-858f-e06b2a380a55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:56.491880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a3637-d5bb-491f-858f-e06b2a380a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:56.493404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a3637-d5bb-491f-858f-e06b2a380a55 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:56.504335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 297/601] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:50:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:50:56.505625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74716bb8-3c3b-4ad9-87d3-d12f8a860281 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.505905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74716bb8-3c3b-4ad9-87d3-d12f8a860281 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.506173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74716bb8-3c3b-4ad9-87d3-d12f8a860281 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.507827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 298/602] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:56.509633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74716bb8-3c3b-4ad9-87d3-d12f8a860281 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:56.512238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30365f-fde0-4718-ac21-9657eedec825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:56.512977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30365f-fde0-4718-ac21-9657eedec825 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.513331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30365f-fde0-4718-ac21-9657eedec825 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.513719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30365f-fde0-4718-ac21-9657eedec825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.535731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:56.594817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30365f-fde0-4718-ac21-9657eedec825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:56.596899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30365f-fde0-4718-ac21-9657eedec825 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:56.619447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:56.650424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74716bb8-3c3b-4ad9-87d3-d12f8a860281 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:56.696888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:56.697973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.698449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.698914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.710390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:56.712210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:56.719024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 305/603] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:50:56.719024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcdca1-13c4-4f83-b7ae-1f61e9c90312 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.719024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcdca1-13c4-4f83-b7ae-1f61e9c90312 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.719024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcdca1-13c4-4f83-b7ae-1f61e9c90312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:56.733452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:56.734434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:56.735325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:56.735325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:56.735325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:56.735929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0bfcdca1-13c4-4f83-b7ae-1f61e9c90312 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:56.736912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0bfcdca1-13c4-4f83-b7ae-1f61e9c90312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:56.737810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 299/604] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:56.739495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:56.795265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:56.829687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 306/605] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:50:56.831408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcdca1-13c4-4f83-b7ae-1f61e9c90312 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:56.855639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe2540-a95f-49e5-914b-4d4be7bf8b93 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:56.886306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:56.893200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:56.894124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:56.894703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:56.898051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:56.909982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:56.912204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:56.940060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:56.941240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:56.942339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:56.942339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:56.942339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:56.947717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 300/606] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:56.956310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.045005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcdca1-13c4-4f83-b7ae-1f61e9c90312 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.092663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.095132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.095633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.096133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.098855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:57.099389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.101002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.114913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 307/607] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:57.122048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.154882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaabb8-50a0-4898-95a5-67e59cdeb0dc None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.205070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:57.235012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.235012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.235012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.235012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.235378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:57.235378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.235378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.241316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 301/608] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:57.242173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.281969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314982-820e-4d9f-a021-400b1c8c5148 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.294057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:57.321765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.322395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.322672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.322941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.330656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.332330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:50:57.343517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:57.344525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:50:57.345450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:57.349913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 308/609] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:57.350759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.357545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ef92e-cc80-47c6-8fc5-b9799a216496 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.392713 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fda88-5316-4652-8015-9769204c6737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.393363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fda88-5316-4652-8015-9769204c6737 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.393622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fda88-5316-4652-8015-9769204c6737 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.393895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fda88-5316-4652-8015-9769204c6737 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.394512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fda88-5316-4652-8015-9769204c6737 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.396029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fda88-5316-4652-8015-9769204c6737 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6c8fda88-5316-4652-8015-9769204c6737 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:57.404938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:57.405888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:50:57.406765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:57.411977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 302/610] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:57.412915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07681e-31d1-427d-b787-a244d2897baf None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.413185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07681e-31d1-427d-b787-a244d2897baf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.413456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07681e-31d1-427d-b787-a244d2897baf None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.414991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 303/611] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:57.415676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07681e-31d1-427d-b787-a244d2897baf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.459300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:57.462918 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb195611-d8a6-4832-ac70-7e975f7da836 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.504605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.505152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.505431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.505706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.512773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.514210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:57.526160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:57.527092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 309/612] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:57.528027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.532952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07681e-31d1-427d-b787-a244d2897baf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.542153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:57.573121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178f8fc-d2a1-40b2-bb0f-6a35740a20a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.573737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178f8fc-d2a1-40b2-bb0f-6a35740a20a9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.574013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178f8fc-d2a1-40b2-bb0f-6a35740a20a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.574341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178f8fc-d2a1-40b2-bb0f-6a35740a20a9 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.575131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178f8fc-d2a1-40b2-bb0f-6a35740a20a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:57.575430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178f8fc-d2a1-40b2-bb0f-6a35740a20a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.576934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178f8fc-d2a1-40b2-bb0f-6a35740a20a9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.598449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 304/613] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:57.599541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f41603-23e4-4021-833b-b71a4516f16f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.599911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f41603-23e4-4021-833b-b71a4516f16f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.600108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f41603-23e4-4021-833b-b71a4516f16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.620175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-42f41603-23e4-4021-833b-b71a4516f16f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:57.621682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-42f41603-23e4-4021-833b-b71a4516f16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:57.645418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c7fce-794c-4ff1-b1bb-c4ea6c6c25ef None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.679548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.680186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.680467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.680745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.681377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:57.681692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.683070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.687424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 310/614] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:57.688225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.691738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 305/615] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:50:57.692347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f41603-23e4-4021-833b-b71a4516f16f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.715739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:57.792954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:57.793760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728bbba-29d3-4419-bfef-94a79eea8aa9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.800031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f41603-23e4-4021-833b-b71a4516f16f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:57.828023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddb320-c47a-4e33-b018-c0eccbdb817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.828640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddb320-c47a-4e33-b018-c0eccbdb817c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.828914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddb320-c47a-4e33-b018-c0eccbdb817c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.829192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddb320-c47a-4e33-b018-c0eccbdb817c None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.829874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddb320-c47a-4e33-b018-c0eccbdb817c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:57.830179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddb320-c47a-4e33-b018-c0eccbdb817c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.832135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddb320-c47a-4e33-b018-c0eccbdb817c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.837370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b2c8b-b946-4dd9-bb11-eec8587f7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:57.837969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b2c8b-b946-4dd9-bb11-eec8587f7027 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.838242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b2c8b-b946-4dd9-bb11-eec8587f7027 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.838508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b2c8b-b946-4dd9-bb11-eec8587f7027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.844296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 311/616] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:57.849897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407e18-a097-42b4-913c-1d78a37f6324 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:57.850557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407e18-a097-42b4-913c-1d78a37f6324 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:57.850836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407e18-a097-42b4-913c-1d78a37f6324 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:57.852667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 312/617] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:50:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:50:57.853567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407e18-a097-42b4-913c-1d78a37f6324 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:57.910626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b2c8b-b946-4dd9-bb11-eec8587f7027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:57.912123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b2c8b-b946-4dd9-bb11-eec8587f7027 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:57.948138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:57.966986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407e18-a097-42b4-913c-1d78a37f6324 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.001805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.002411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.002693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.002971 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.003699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:58.003864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:58.004050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.005412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.010015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 313/618] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:58.010803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.016384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 306/619] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:50:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:50:58.016384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4076f1-55f9-4899-b155-54ae0f8be1a5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.016384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4076f1-55f9-4899-b155-54ae0f8be1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.016384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4076f1-55f9-4899-b155-54ae0f8be1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.037427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2b4076f1-55f9-4899-b155-54ae0f8be1a5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:50:58.038533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2b4076f1-55f9-4899-b155-54ae0f8be1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:50:58.116695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 307/620] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:50:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:50:58.117663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4076f1-55f9-4899-b155-54ae0f8be1a5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.154291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806c590-dca6-4b4b-b771-4c23c689d8e6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.201225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.202094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.202570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.203274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.203562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=6bb4908b26e64c538c2b573f3cf26bf1) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:58.204227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.205651 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.214501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 314/621] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/role_assignments?user.id=6bb4908b26e64c538c2b573f3cf26bf1&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:58.215189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.235668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:58.267096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4076f1-55f9-4899-b155-54ae0f8be1a5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.298115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:58.313462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.315512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.316934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.317452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.318417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.321215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:58.338195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:58.339262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 10:50:58.340179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:50:58.344668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 308/622] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:58.356063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.395643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fffbe-a2c3-4f3f-842b-ef06451f4c6a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.432965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.433776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.434172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.434542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.446019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.447522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:50:58.462942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:58.464200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:50:58.465296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:50:58.465296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:50:58.465296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:58.465296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 315/623] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:58.465296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.498990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3f739-e1bf-4009-9a92-3a9504572bb1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.536645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.537301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.537527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.537822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.538427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] RBAC: Adding query filter params (name=network) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:58.538731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.540370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.545699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 309/624] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:58.546612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.552638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:58.594368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfde24-93ef-4be0-9f61-b168e06fd909 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.609190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:58.630671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.631306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.631582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.631881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.632663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.634102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:58.645037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:58.646303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:58.647369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 316/625] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:58.648506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.680029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048862bc-4774-4371-9d6f-42f9ab914f08 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.723545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.724217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.724753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.725023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.726194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:58.726638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.728079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.748211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 310/626] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:58.749216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.770592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1378149-effe-437c-9ecd-2131ebb8aa4d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.801270 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:58.810392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.813966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.813966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.813966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.813966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] RBAC: Adding query filter params (type=network) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:58.813966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.814906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.823881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 317/627] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:58.824762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.868620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca372b-c60b-45b4-b1c6-bd6b2d24b877 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.883953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:58.906735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.907340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.907619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.907971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:58.908586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:58.908880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:58.911052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:58.916085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 311/628] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:58.916965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:58.948213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace0cba-dd68-4436-8927-c45e3d9da72b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:58.990728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:58.991413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:58.991694 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:58.992047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:59.000342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:59.001375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:50:59.014391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:50:59.015329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:50:59.016274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:50:59.019863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 318/629] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:50:59.020881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:59.053501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd64426-b597-4b57-968c-2e9088c8e181 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:59.086885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:59.102469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:59.103592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:59.104838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:59.105328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:59.106251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:59.107906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:59.136286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec52f-fa9b-46e5-9b5f-4208c7377793 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:59.171283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:59.171843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:59.172123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:59.173363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:59.173363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:59.173363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:59.174632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:59.180563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 319/630] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 26 10:50:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:59.181406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:59.240678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 312/631] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:50:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:59.242519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:59.296283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:59.303859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143f50e-9358-403b-9c3f-4070f152b845 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:59.337431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:50:59.394146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30c1f0-2834-4598-aeff-68dd59e6d604 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:59.409168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:59.410326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:59.410604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:59.410871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:59.411572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:59.411901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:59.413323 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:59.436531 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 320/632] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:50:59.444911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:50:59.445882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef444f-40c7-4c24-933d-dd497a92d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:59.446777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef444f-40c7-4c24-933d-dd497a92d639 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:59.447260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef444f-40c7-4c24-933d-dd497a92d639 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:59.447858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef444f-40c7-4c24-933d-dd497a92d639 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:59.448884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef444f-40c7-4c24-933d-dd497a92d639 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:59.450861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef444f-40c7-4c24-933d-dd497a92d639 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:59.540547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 313/633] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:50:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:50:59.590733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac266a83-4cb2-423f-a465-0d9e77bcc1e9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:50:59.637423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865fb9b-6eaa-4aa0-a019-d60aeafed72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:50:59.638424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865fb9b-6eaa-4aa0-a019-d60aeafed72f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:50:59.638925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865fb9b-6eaa-4aa0-a019-d60aeafed72f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:50:59.639392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865fb9b-6eaa-4aa0-a019-d60aeafed72f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:50:59.640365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865fb9b-6eaa-4aa0-a019-d60aeafed72f None admin] RBAC: Adding query filter params (role.id=7f166188476b48158de248c89d0b9144, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=d64e1522d1d4425bbb88d78e4f8863d6) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:50:59.640874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865fb9b-6eaa-4aa0-a019-d60aeafed72f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:50:59.642692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865fb9b-6eaa-4aa0-a019-d60aeafed72f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:50:59.650029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 321/634] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:50:59 2022] GET /identity/v3/role_assignments?user.id=d64e1522d1d4425bbb88d78e4f8863d6&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=7f166188476b48158de248c89d0b9144 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:00.382525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a59b34-0b90-4890-b9c7-afa966421d54 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:00.382908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a59b34-0b90-4890-b9c7-afa966421d54 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:00.383102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a59b34-0b90-4890-b9c7-afa966421d54 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:00.384927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 314/635] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:00.390290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193e43-c3b4-4b48-842a-124380d8e9d9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:00.390582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193e43-c3b4-4b48-842a-124380d8e9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:00.390859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193e43-c3b4-4b48-842a-124380d8e9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:00.410294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f4193e43-c3b4-4b48-842a-124380d8e9d9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:00.411682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f4193e43-c3b4-4b48-842a-124380d8e9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:00.439642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:00.486641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 322/636] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:00.619151 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588753e2-7e53-4fd1-a58b-093304e23db5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:00.619670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588753e2-7e53-4fd1-a58b-093304e23db5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:00.619778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588753e2-7e53-4fd1-a58b-093304e23db5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:00.621378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 315/637] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:00.627268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71bdf-4233-4a77-ae98-657feb33694a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:00.627764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71bdf-4233-4a77-ae98-657feb33694a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:00.627876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71bdf-4233-4a77-ae98-657feb33694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:00.647518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f6a71bdf-4233-4a77-ae98-657feb33694a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:00.648674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f6a71bdf-4233-4a77-ae98-657feb33694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:00.722537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 323/638] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:00.760377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588753e2-7e53-4fd1-a58b-093304e23db5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:00.853261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:00.873442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588753e2-7e53-4fd1-a58b-093304e23db5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:00.909311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:00.909970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:00.910335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:00.910610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:00.911262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:00.912775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:01.001969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a71bdf-4233-4a77-ae98-657feb33694a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:01.004002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 316/639] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:00 2022] POST /identity/v3/services => generated 269 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:01.125009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:01.135805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a71bdf-4233-4a77-ae98-657feb33694a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:01.181515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:01.182495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:01.182972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:01.183440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:01.184552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:01.186111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:01.195569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:01.196554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:01.197397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:01.198676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 324/640] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:01.203772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:01.317208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc57074-44d5-4ef9-90f6-a276eea00e14 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:01.353235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903966c5-a5f7-4fea-a9fa-3bb51e0e643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:01.354229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903966c5-a5f7-4fea-a9fa-3bb51e0e643b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:01.354721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903966c5-a5f7-4fea-a9fa-3bb51e0e643b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:01.355234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903966c5-a5f7-4fea-a9fa-3bb51e0e643b None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:01.356121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903966c5-a5f7-4fea-a9fa-3bb51e0e643b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:01.356677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903966c5-a5f7-4fea-a9fa-3bb51e0e643b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:01.358289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903966c5-a5f7-4fea-a9fa-3bb51e0e643b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:01.364359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 317/641] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:01.370199 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:01.476086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ea9e-0f3f-4088-8059-34c9220eb492 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:01.514274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2b566-3a4b-47df-8e85-c0524de26d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:01.514880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b566-3a4b-47df-8e85-c0524de26d61 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:01.515158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b566-3a4b-47df-8e85-c0524de26d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:01.515426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b566-3a4b-47df-8e85-c0524de26d61 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:01.516070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2b566-3a4b-47df-8e85-c0524de26d61 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:01.516371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2b566-3a4b-47df-8e85-c0524de26d61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:01.517720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2b566-3a4b-47df-8e85-c0524de26d61 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:01.522735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 325/642] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:01.847807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f3e11-9e43-4e0b-8383-8528370c559e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:01.848340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f3e11-9e43-4e0b-8383-8528370c559e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:01.848420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f3e11-9e43-4e0b-8383-8528370c559e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:01.850165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 318/643] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:01.856376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29c4a4-f74f-4523-ba76-43d6d7bd8ad0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:01.856920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29c4a4-f74f-4523-ba76-43d6d7bd8ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:01.856920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29c4a4-f74f-4523-ba76-43d6d7bd8ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:01.875763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5a29c4a4-f74f-4523-ba76-43d6d7bd8ad0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:01.876974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5a29c4a4-f74f-4523-ba76-43d6d7bd8ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:01.889796 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:01.953470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 326/644] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:01.955290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350096-8fad-446b-9948-523c9a14871a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:01.955585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350096-8fad-446b-9948-523c9a14871a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:01.955892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350096-8fad-446b-9948-523c9a14871a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:01.957511 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 319/645] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:01.962661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707986df-f1b6-43ea-85e7-08ab55e744e3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:01.962953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707986df-f1b6-43ea-85e7-08ab55e744e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:01.963232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707986df-f1b6-43ea-85e7-08ab55e744e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:01.982932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-707986df-f1b6-43ea-85e7-08ab55e744e3 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:01.984137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-707986df-f1b6-43ea-85e7-08ab55e744e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:02.071175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 327/646] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:02.224407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84350096-8fad-446b-9948-523c9a14871a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:02.238093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:02.332736 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84350096-8fad-446b-9948-523c9a14871a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:02.336140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707986df-f1b6-43ea-85e7-08ab55e744e3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:02.375556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:02.376220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:02.376500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:02.376769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:02.377885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:02.379328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:02.391872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:02.392860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:02.394263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:02.395872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 320/647] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:02.400762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:02.424225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:02.451778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707986df-f1b6-43ea-85e7-08ab55e744e3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:02.495615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:02.496781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:02.497209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:02.497515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:02.498795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:02.512463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:02.519015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:02.519985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:02.520823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:02.523239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 328/648] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 26 10:51:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:02.530195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:02.533183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a757de-11aa-4c5a-83ad-7db0601e8986 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:02.584227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:02.585105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:02.585550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:02.585983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:02.586753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:02.587738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:02.589601 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:02.602189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 321/649] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:02.603734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:02.652377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f5e2e-d7ce-43d9-a0f8-962e168d2be4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:02.698312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:02.698922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:02.699252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:02.699590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:02.700274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:02.700578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:02.701947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:02.704500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:02.716074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 329/650] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 26 10:51:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:02.717534 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:02.720471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c207e62-a4fa-432f-96ba-64e5a323c5d9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:02.763585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:02.764632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:02.765118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:02.765611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:02.766554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:02.767153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:02.768825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:02.771430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:02.783117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 322/651] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:02.829706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ee00-7e42-41ea-8d8f-4ac96275f132 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:02.869802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:02.870466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:02.870717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:02.871035 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:02.883615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:02.885372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:02.901673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:02.902677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:02.904042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:02.904042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:02.904042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:02.904042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 330/652] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:02.908917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:03.022128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246022d3-5050-46a6-a34b-f32887796d8f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:03.063689 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:03.064589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.065011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.065440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.067314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:03.067762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:03.069281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:03.082581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 323/653] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:03.087871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:03.140248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:03.192814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2e2ed-1c34-4c01-9a31-679099c66604 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:03.227925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:03.228563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.228844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.229110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.237655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:03.243101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:51:03.251454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:03.252646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:03.255029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 331/654] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:51:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:03.259249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:03.330174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:03.370013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98026dc2-c751-4e21-9b69-ee4cceeaabe2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:03.408171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43faff5d-e9a2-4950-9585-842b367ac66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:03.408781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faff5d-e9a2-4950-9585-842b367ac66d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.409112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faff5d-e9a2-4950-9585-842b367ac66d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.409424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faff5d-e9a2-4950-9585-842b367ac66d None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.410214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43faff5d-e9a2-4950-9585-842b367ac66d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:03.410547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43faff5d-e9a2-4950-9585-842b367ac66d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:03.412112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43faff5d-e9a2-4950-9585-842b367ac66d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:03.430524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 324/655] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 26 10:51:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:03.436125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:03.477474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618855b-6313-4dc1-a49c-ecd82c6d2225 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.477798 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618855b-6313-4dc1-a49c-ecd82c6d2225 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.478028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618855b-6313-4dc1-a49c-ecd82c6d2225 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.479495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 325/656] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:51:03.484688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebca9f-9126-413f-8a88-985e720af18f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.484970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebca9f-9126-413f-8a88-985e720af18f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.485243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebca9f-9126-413f-8a88-985e720af18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.506622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6eebca9f-9126-413f-8a88-985e720af18f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:03.508590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6eebca9f-9126-413f-8a88-985e720af18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:03.546214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da67c-87fb-41d9-a503-5001fbf0b22d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:03.558139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:03.584944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be85bf-ab1a-4329-8aa8-2aba7ec44993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:03.588409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be85bf-ab1a-4329-8aa8-2aba7ec44993 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.588409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be85bf-ab1a-4329-8aa8-2aba7ec44993 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.588409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be85bf-ab1a-4329-8aa8-2aba7ec44993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.592361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 326/657] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:03.604657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be85bf-ab1a-4329-8aa8-2aba7ec44993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:03.606467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be85bf-ab1a-4329-8aa8-2aba7ec44993 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c9be85bf-ab1a-4329-8aa8-2aba7ec44993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:03.622912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:03.623922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:03.624862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:03.624862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:03.624862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:03.626906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 332/658] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:03.635842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebca9f-9126-413f-8a88-985e720af18f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:03.696585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:03.741263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebca9f-9126-413f-8a88-985e720af18f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:03.772964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075687-35c8-46e3-81f2-05235cea5297 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.773287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075687-35c8-46e3-81f2-05235cea5297 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.773519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075687-35c8-46e3-81f2-05235cea5297 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.775160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 333/659] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:51:03.780195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:03.780816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.780921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b540d9e-c079-4a50-bc59-6d8e8e13579c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.781087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.781181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b540d9e-c079-4a50-bc59-6d8e8e13579c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.781376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.781479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b540d9e-c079-4a50-bc59-6d8e8e13579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.782028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:03.782334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:03.783740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:03.798541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 327/660] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:03.800535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4b540d9e-c079-4a50-bc59-6d8e8e13579c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:03.801846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4b540d9e-c079-4a50-bc59-6d8e8e13579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:03.804270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:03.907810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 334/661] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:03.907810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b540d9e-c079-4a50-bc59-6d8e8e13579c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:03.928106 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a94b-341a-4dbf-a1d2-88a9c7cafcf7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:03.940147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:03.975466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:03.976380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:03.976662 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:03.977350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:03.987660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:03.990032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:51:04.005218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:04.006137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:04.007063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 328/662] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:04.016725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:04.018852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:04.029377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b540d9e-c079-4a50-bc59-6d8e8e13579c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.084192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.084867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.085150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.085427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.086287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.087953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:04.100824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:04.101839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:04.102738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 335/663] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 26 10:51:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:04.107746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:04.163916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8c55e-ccd0-4ccf-97e0-4c9202cca038 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.204162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.204829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.205189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.205468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.206312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:04.206629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.208049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.221398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 329/664] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:04.222437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeb5da-bd70-4ef4-be60-64ad8789e5c5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.222702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:04.256444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:04.264479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.265085 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.265420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.265691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.266345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:04.266650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.268024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.273913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 336/665] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:04.274738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:04.299201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:04.338015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b709ca-2a43-4b72-8201-7cc06bb5a097 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.374287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.374891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.375168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.375439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.376129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.377481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.382252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba13a3-87b9-405d-a87f-4c18b3cbb837 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.417692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.418279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.418555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.418827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/d64e1522d1d4425bbb88d78e4f8863d6/roles/780ae8b185504927a727c332e2260a6d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.420955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=d64e1522d1d4425bbb88d78e4f8863d6, role_id=780ae8b185504927a727c332e2260a6d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.422374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.455419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 330/666] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:04 2022] POST /identity/v3/services => generated 241 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:04.457194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:04.533979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 337/667] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:51:04 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/d64e1522d1d4425bbb88d78e4f8863d6/roles/780ae8b185504927a727c332e2260a6d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:51:04.580306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:04.602054 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5cfe4-01a9-4931-996c-38c7f7dff1f3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.644647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.645357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.645646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.645920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.646567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:04.648999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.648999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.654651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 331/668] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:04.661771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:04.761882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:04.781683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f0c1-227f-427e-8ce8-570a8151bc2c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.821519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00904584-c93d-4eca-aa39-b2e3e3988afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.822428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00904584-c93d-4eca-aa39-b2e3e3988afe None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.822882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00904584-c93d-4eca-aa39-b2e3e3988afe None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.823324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00904584-c93d-4eca-aa39-b2e3e3988afe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.824217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00904584-c93d-4eca-aa39-b2e3e3988afe None admin] RBAC: Adding query filter params (interface=public, service_id=31496bc6babd4f7c8a31bb50bf8091bd, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:04.824746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00904584-c93d-4eca-aa39-b2e3e3988afe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.826380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00904584-c93d-4eca-aa39-b2e3e3988afe None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.832908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 338/669] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:51:04 2022] GET /identity/v3/endpoints?service_id=31496bc6babd4f7c8a31bb50bf8091bd&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:04.839906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:04.947944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b097672-026d-43ba-808b-0321f9213fa0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:04.985023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd097-bbf2-4f4a-9f91-4d1882b1e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:04.985656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd097-bbf2-4f4a-9f91-4d1882b1e289 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:04.986050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd097-bbf2-4f4a-9f91-4d1882b1e289 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:04.986325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd097-bbf2-4f4a-9f91-4d1882b1e289 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:04.986947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cd097-bbf2-4f4a-9f91-4d1882b1e289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:04.989334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cd097-bbf2-4f4a-9f91-4d1882b1e289 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:04.999982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 332/670] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:51:04 2022] GET /identity/v3/services => generated 1258 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 10:51:05.116630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f74e-7ee5-4319-8133-dac8198af8fc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:05.117620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f74e-7ee5-4319-8133-dac8198af8fc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:05.118153 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f74e-7ee5-4319-8133-dac8198af8fc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:05.120034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 339/671] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:05.126161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f81de4-6fb9-4972-8934-b6b72a8f24f0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:05.126744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f81de4-6fb9-4972-8934-b6b72a8f24f0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:05.127234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f81de4-6fb9-4972-8934-b6b72a8f24f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:05.147637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-57f81de4-6fb9-4972-8934-b6b72a8f24f0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:05.149402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-57f81de4-6fb9-4972-8934-b6b72a8f24f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:05.158410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:05.222595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 333/672] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:05 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:05.492492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f74e-7ee5-4319-8133-dac8198af8fc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:05.605485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f74e-7ee5-4319-8133-dac8198af8fc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:05.640489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c71aed-accc-4875-865c-0fa381f4ff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:05.641089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c71aed-accc-4875-865c-0fa381f4ff41 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:05.641364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c71aed-accc-4875-865c-0fa381f4ff41 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:05.641631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c71aed-accc-4875-865c-0fa381f4ff41 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:05.642279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c71aed-accc-4875-865c-0fa381f4ff41 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:05.643683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c71aed-accc-4875-865c-0fa381f4ff41 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:05.720462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 340/673] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:05 2022] POST /identity/v3/services => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:06.811545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163088a-1ba8-40d1-8b08-a516c19cae6f None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:06.812098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163088a-1ba8-40d1-8b08-a516c19cae6f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:06.812469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163088a-1ba8-40d1-8b08-a516c19cae6f None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:06.813969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 334/674] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:06.814078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f214-4472-4685-a1d9-97ecaf530e5c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:06.814428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f214-4472-4685-a1d9-97ecaf530e5c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:06.814500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f214-4472-4685-a1d9-97ecaf530e5c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:06.816383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 341/675] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:51:06.818989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11f2b5-abba-4a4d-a15c-dcb62f5e3747 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:06.819526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11f2b5-abba-4a4d-a15c-dcb62f5e3747 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:06.820069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11f2b5-abba-4a4d-a15c-dcb62f5e3747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:06.823873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255a83e-2a5b-4f63-bf85-800b26f1765e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:06.824159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255a83e-2a5b-4f63-bf85-800b26f1765e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:06.824429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255a83e-2a5b-4f63-bf85-800b26f1765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:06.832247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:06.844253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3255a83e-2a5b-4f63-bf85-800b26f1765e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:06.844420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9c11f2b5-abba-4a4d-a15c-dcb62f5e3747 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:06.845552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9c11f2b5-abba-4a4d-a15c-dcb62f5e3747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:06.848465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3255a83e-2a5b-4f63-bf85-800b26f1765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:06.936584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 342/676] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:06.937363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 335/677] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:07.209172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255a83e-2a5b-4f63-bf85-800b26f1765e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.211082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11f2b5-abba-4a4d-a15c-dcb62f5e3747 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.299200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:07.319255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255a83e-2a5b-4f63-bf85-800b26f1765e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:07.332606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11f2b5-abba-4a4d-a15c-dcb62f5e3747 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:07.350373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:07.356288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e562495-171d-46dd-858c-70aeb5a5fbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:07.357150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e562495-171d-46dd-858c-70aeb5a5fbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.357433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e562495-171d-46dd-858c-70aeb5a5fbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.357704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e562495-171d-46dd-858c-70aeb5a5fbb9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.358284 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e562495-171d-46dd-858c-70aeb5a5fbb9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:07.359646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e562495-171d-46dd-858c-70aeb5a5fbb9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1e562495-171d-46dd-858c-70aeb5a5fbb9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:07.368399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:07.377258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:07.378515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 343/678] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:07.379508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dc80d-bf20-4943-a2df-3bb291f47d8f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.379508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dc80d-bf20-4943-a2df-3bb291f47d8f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.379508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dc80d-bf20-4943-a2df-3bb291f47d8f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.379508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 344/679] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:07.379508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dc80d-bf20-4943-a2df-3bb291f47d8f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.381206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746e18c-50c8-49b8-b720-85e99f1681fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:07.381860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746e18c-50c8-49b8-b720-85e99f1681fa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.382143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746e18c-50c8-49b8-b720-85e99f1681fa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.382417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746e18c-50c8-49b8-b720-85e99f1681fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.396130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746e18c-50c8-49b8-b720-85e99f1681fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e746e18c-50c8-49b8-b720-85e99f1681fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:07.401034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:07.402033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:07.402869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 336/680] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:07.403546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf946a-b4eb-45f9-8145-6d0877d233ff None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.403546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf946a-b4eb-45f9-8145-6d0877d233ff None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.404216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf946a-b4eb-45f9-8145-6d0877d233ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.422566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fbcf946a-b4eb-45f9-8145-6d0877d233ff None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:07.423752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fbcf946a-b4eb-45f9-8145-6d0877d233ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:07.491158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dc80d-bf20-4943-a2df-3bb291f47d8f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:07.503820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 337/681] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:07.503820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf946a-b4eb-45f9-8145-6d0877d233ff None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.527478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:07.528147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.528428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.528708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.529335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:07.529638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:07.531009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:07.541388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 345/682] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:07.543351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.579343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:07.614699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:07.625769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf946a-b4eb-45f9-8145-6d0877d233ff None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:07.665025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f731-5bb7-4b3a-b053-cd23012835e0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:07.680026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:07.680655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.680920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.681190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.682225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:07.682526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:07.684191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:07.695610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 338/683] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:07.702327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.710750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:07.711619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.712151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.712625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.713860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:07.714366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:07.716294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:07.723679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 346/684] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:07.729166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.824965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106a556-d012-403c-b293-6d5f0c38c72d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:07.845008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e106945-01a3-47bc-95a0-1ba6f056a687 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:07.851956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:07.865843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:07.866890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.867410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.868082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.871338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:07.883929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:07.886094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:07.886258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:07.886910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:07.887197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:07.887478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:07.888257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] RBAC: Adding query filter params (interface=public, service_id=130340b3022242d99dc8f19ee54be28b, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:07.888567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:07.890157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:07.896301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 347/685] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/endpoints?service_id=130340b3022242d99dc8f19ee54be28b&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:07.897215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:07.907440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:07.908644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:07.910225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:07.910225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:07.910225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:07.912436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 339/686] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:07.914540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.012353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afcbd1-9528-4449-a442-4f79526d4209 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.017812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e96a14-0f20-4932-9888-93f1791d1635 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.047089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.047722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.048025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.048307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.048871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.051002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.052797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c941c-1f39-4cd1-94e5-7932373bdaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.053402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c941c-1f39-4cd1-94e5-7932373bdaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.053684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c941c-1f39-4cd1-94e5-7932373bdaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.053957 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c941c-1f39-4cd1-94e5-7932373bdaf1 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.054555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c941c-1f39-4cd1-94e5-7932373bdaf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.055949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c941c-1f39-4cd1-94e5-7932373bdaf1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:08.059637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:08.060652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:08.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 348/687] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:08.066140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.066221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 340/688] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:51:07 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 10:51:08.066221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ade4c1-73d1-46a4-8271-0448abc86cdc None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.066221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ade4c1-73d1-46a4-8271-0448abc86cdc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.066221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ade4c1-73d1-46a4-8271-0448abc86cdc None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.066221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 341/689] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:08.071832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ade4c1-73d1-46a4-8271-0448abc86cdc None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.127485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:08.129333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:08.185733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087a65e-5edc-43d3-8efd-7011e995acb0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.196268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ade4c1-73d1-46a4-8271-0448abc86cdc None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.226494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.227524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.228113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.228391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.229047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:08.229537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.230967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.233560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e606d34-b643-4e83-9fdd-dc863bfa6529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.234175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e606d34-b643-4e83-9fdd-dc863bfa6529 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.234458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e606d34-b643-4e83-9fdd-dc863bfa6529 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.234727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e606d34-b643-4e83-9fdd-dc863bfa6529 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.235429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e606d34-b643-4e83-9fdd-dc863bfa6529 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:08.235729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e606d34-b643-4e83-9fdd-dc863bfa6529 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.243274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e606d34-b643-4e83-9fdd-dc863bfa6529 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.243274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 342/690] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:08.244362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800fcb4-8528-4bff-a619-8d8722bbce9c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.245357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800fcb4-8528-4bff-a619-8d8722bbce9c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.245357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800fcb4-8528-4bff-a619-8d8722bbce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.251880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 349/691] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:08.253231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.266246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d800fcb4-8528-4bff-a619-8d8722bbce9c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:08.267441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d800fcb4-8528-4bff-a619-8d8722bbce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:08.351963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 343/692] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:08.351963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800fcb4-8528-4bff-a619-8d8722bbce9c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.372706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bc888-070f-491a-8f20-96f255996514 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.385526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:08.388295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:08.411674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.412733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.413048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.413415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.414145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:08.414539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.416114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.421498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 350/693] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:08.428236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.475811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800fcb4-8528-4bff-a619-8d8722bbce9c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.525308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.526065 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.526415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.527009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.537336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.539647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.545146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4523-8ed4-47de-ba18-9f754d5cef85 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:51:08.554274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:08.555275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 10:51:08.556613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:08.559929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 344/694] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:08.561817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.595664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.596315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.596624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.596894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.597559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.599404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.663868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:08.670277 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46036579-197d-443b-89b3-1b7d1d49ffaa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.703808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.704419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.704767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.705040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.706492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:08.706845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.712354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 351/695] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:51:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:08.712538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.712621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.739181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 345/696] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:08.746169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.754988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:08.853809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254ec4c-95ce-4019-bd7a-a84dcbb94260 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.869091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd451a-f10b-4cd5-8edb-26c95c4ace2b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:08.896319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.897092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.897443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.897784 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.898437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:08.900602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:08.911353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:08.911563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:08.912538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:08.913394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:08.914178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:08.914178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:08.914314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 352/697] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:08.917414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:08.919264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:08.944005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:09.001669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.003249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.045486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:09.057746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c2ea1-441b-4682-a5fe-6afc4586ddc6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.103813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.104607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.104988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.105362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.106087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:09.106492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.108509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.123359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 353/698] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:09.123359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.144313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 346/699] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:51:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 26 10:51:09.151468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.209406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:09.261211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24d48-6c10-4200-927a-984ae0106115 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.270964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a7c56-d45f-482d-ba7f-12154dd00c0f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.301516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.302265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.302557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.302836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.317678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.318655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.319338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.319690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.319814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.320053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.320744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:09.321113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.322550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.327819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 354/700] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:09.333662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:09.340497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:09.341450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:09.342361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:09.342361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:09.342361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:09.346961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 347/701] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:09.348489 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.359447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:09.455615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539f151-be49-42b1-a26d-73ec4bdacac1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.489549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d988a60-fdf5-4dc3-8783-cce500e13b92 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.492054 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:09.502012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.510881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.510881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.510881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.510881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] RBAC: Adding query filter params (interface=public, service_id=1dfb8a18368144458ddee33915c04ac7, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:09.510881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.510881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.514214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 355/702] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/endpoints?service_id=1dfb8a18368144458ddee33915c04ac7&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:09.520174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.530312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.530925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.531751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.532072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.532724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:09.533031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.535092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.551922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 348/703] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:09.557611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.622050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:09.645937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f3071-4baf-4ad5-a825-0acaa77e03b4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.672767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61ffee-1e1f-41da-a704-e3699b9a55c6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.691091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.692204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.692491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.692765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.693379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.695276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.703001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 356/704] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 10:51:09.711704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.712411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.712687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.712954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.721059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.723106 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:51:09.736344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:09.739314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:09.740597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 349/705] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:09.745120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.864246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1444f8-dd38-4173-82d2-11620eae908d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:09.902955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f28222-24d9-474c-8442-a85add9505ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:09.903571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f28222-24d9-474c-8442-a85add9505ad None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:09.903882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f28222-24d9-474c-8442-a85add9505ad None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:09.904159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f28222-24d9-474c-8442-a85add9505ad None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:09.904895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f28222-24d9-474c-8442-a85add9505ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:09.905159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f28222-24d9-474c-8442-a85add9505ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:09.906585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f28222-24d9-474c-8442-a85add9505ad None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:09.920506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 357/706] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:09.927028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:09.955614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:10.038064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7edac-9288-4aab-b0ab-36f671462862 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:10.078305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68909e-b352-4c58-bc37-f55c9e759330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:10.078977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68909e-b352-4c58-bc37-f55c9e759330 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:10.079259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68909e-b352-4c58-bc37-f55c9e759330 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:10.079575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68909e-b352-4c58-bc37-f55c9e759330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:10.080344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68909e-b352-4c58-bc37-f55c9e759330 None admin] RBAC: Adding query filter params (role.id=780ae8b185504927a727c332e2260a6d, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=d64e1522d1d4425bbb88d78e4f8863d6) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:10.080654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68909e-b352-4c58-bc37-f55c9e759330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:10.082188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68909e-b352-4c58-bc37-f55c9e759330 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:10.089400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 350/707] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:51:09 2022] GET /identity/v3/role_assignments?user.id=d64e1522d1d4425bbb88d78e4f8863d6&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=780ae8b185504927a727c332e2260a6d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:10.391208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac01421-0841-472a-a962-819a233ac9f5 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:10.391589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac01421-0841-472a-a962-819a233ac9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:10.391769 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac01421-0841-472a-a962-819a233ac9f5 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:10.394427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 358/708] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:10.399624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234c813-67f5-4ef4-a99f-efec54944346 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:10.399942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234c813-67f5-4ef4-a99f-efec54944346 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:10.400211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234c813-67f5-4ef4-a99f-efec54944346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:10.420020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6234c813-67f5-4ef4-a99f-efec54944346 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:10.421219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6234c813-67f5-4ef4-a99f-efec54944346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:10.492004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 351/709] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:10.772939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac01421-0841-472a-a962-819a233ac9f5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:10.782467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:10.882869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac01421-0841-472a-a962-819a233ac9f5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:10.918628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12683982-2746-46aa-84a0-b52197c545cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:10.919267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683982-2746-46aa-84a0-b52197c545cf None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:10.919548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683982-2746-46aa-84a0-b52197c545cf None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:10.919845 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683982-2746-46aa-84a0-b52197c545cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:10.920432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683982-2746-46aa-84a0-b52197c545cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:10.921792 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683982-2746-46aa-84a0-b52197c545cf None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-12683982-2746-46aa-84a0-b52197c545cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:10.931072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:10.932115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:10.933555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 359/710] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:10.941697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234c813-67f5-4ef4-a99f-efec54944346 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:10.972224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:11.052797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234c813-67f5-4ef4-a99f-efec54944346 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:11.088544 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:11.089150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.089431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.089699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.090507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:11.090811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:11.092239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:11.097475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 352/711] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:11.098279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddd9c7-8b85-4505-8b1c-ee454088d9d4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.098577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddd9c7-8b85-4505-8b1c-ee454088d9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.098855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddd9c7-8b85-4505-8b1c-ee454088d9d4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.100443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 360/712] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:11.102720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:11.107808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ee320-466f-46f0-8bd7-977a4b3d4589 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.107808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ee320-466f-46f0-8bd7-977a4b3d4589 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.107808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ee320-466f-46f0-8bd7-977a4b3d4589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.126218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e34ee320-466f-46f0-8bd7-977a4b3d4589 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:11.127410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e34ee320-466f-46f0-8bd7-977a4b3d4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:11.199280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:11.202781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:11.208924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 361/713] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:11.218787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3366b0-d02a-4824-9580-9d7909c61e50 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:11.255588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:11.256233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.256517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.256792 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.257426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:11.257730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:11.259110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:11.264810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 353/714] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:11.271100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ee320-466f-46f0-8bd7-977a4b3d4589 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:11.381530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ee320-466f-46f0-8bd7-977a4b3d4589 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:11.417910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:11.418867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.419382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.419877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.420739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:11.422362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:11.479687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:11.503871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 362/715] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:51:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:11.608165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6813fa-abc9-44c1-821a-5762636484a6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:11.653300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:11.653856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.654133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.654468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:11.656454 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:11.670693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:11.671774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:11.672646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:11.675696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 354/716] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:11.682076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:11.743763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:11.791119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e01fcb-a1a2-4548-bee1-052b0c222179 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:11.827817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed974c3-09d8-41e7-a9ad-83f3a43562be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:11.828445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed974c3-09d8-41e7-a9ad-83f3a43562be None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.828721 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed974c3-09d8-41e7-a9ad-83f3a43562be None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.829088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed974c3-09d8-41e7-a9ad-83f3a43562be None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.829719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed974c3-09d8-41e7-a9ad-83f3a43562be None admin] RBAC: Adding query filter params (name=compute) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:11.830020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed974c3-09d8-41e7-a9ad-83f3a43562be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:11.832069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed974c3-09d8-41e7-a9ad-83f3a43562be None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:11.839903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 363/717] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:51:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:11.843009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:11.852815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:11.957558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374040b5-58b0-4be8-84d4-3a5bb6908434 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:11.997171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a4e72-eb66-4321-8f06-a71d81b3a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:11.998056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a4e72-eb66-4321-8f06-a71d81b3a3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:11.998478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a4e72-eb66-4321-8f06-a71d81b3a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:11.998892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a4e72-eb66-4321-8f06-a71d81b3a3e9 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:11.999651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a4e72-eb66-4321-8f06-a71d81b3a3e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:12.000123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a4e72-eb66-4321-8f06-a71d81b3a3e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:12.001699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a4e72-eb66-4321-8f06-a71d81b3a3e9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:12.007156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 355/718] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:51:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:12.074919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d1a8-5bd9-41d7-b916-4d60862e78d3 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.075942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d1a8-5bd9-41d7-b916-4d60862e78d3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.076423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d1a8-5bd9-41d7-b916-4d60862e78d3 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.078379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 364/719] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:12.084914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a253f35-e1a6-4b51-9c89-a6cd331479b6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.085261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a253f35-e1a6-4b51-9c89-a6cd331479b6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.085477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a253f35-e1a6-4b51-9c89-a6cd331479b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.105227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0a253f35-e1a6-4b51-9c89-a6cd331479b6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:12.106457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0a253f35-e1a6-4b51-9c89-a6cd331479b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:12.181210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:12.187674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 356/720] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:12.345333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f48a5-fd20-4391-8be2-fb7026184818 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.345632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f48a5-fd20-4391-8be2-fb7026184818 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.345813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f48a5-fd20-4391-8be2-fb7026184818 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.348061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 365/721] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:12.353616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7e80-185b-483a-8c3d-4c1e4f554023 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.353910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7e80-185b-483a-8c3d-4c1e4f554023 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.354191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7e80-185b-483a-8c3d-4c1e4f554023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.373354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-092c7e80-185b-483a-8c3d-4c1e4f554023 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:12.374548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-092c7e80-185b-483a-8c3d-4c1e4f554023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:12.444633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 357/722] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:12.479446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f48a5-fd20-4391-8be2-fb7026184818 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:12.587229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f48a5-fd20-4391-8be2-fb7026184818 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:12.607523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:12.621949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:12.622620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.622957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.630331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.630331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:12.630331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:12.634559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:12.635747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:12.637063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 366/723] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:12.641936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7e80-185b-483a-8c3d-4c1e4f554023 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:12.744077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:12.747994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7e80-185b-483a-8c3d-4c1e4f554023 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:12.779639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:12.790590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:12.791577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.792101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.792579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.793414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:12.793917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:12.796314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:12.811281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 358/724] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:12.812835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:12.884577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b9e8e-8dfd-4a75-8170-ad47ba21d1ef None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:12.924995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:12.925663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.925941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.926211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.927060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:12.929163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:12.931485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8229d0c-797a-43e9-9683-14638f7ae5f9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:12.938714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:12.939724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:12.945713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 367/725] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Oct 26 10:51:12 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:12.947687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:12.976288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:12.977576 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:12.978000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:12.978505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:12.979315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:12.979775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:12.981506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:12.987978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 359/726] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 26 10:51:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:12.997827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:13.015709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:13.085141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:13.085630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ed237-61e6-4549-8529-a116006a2ad5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:13.132923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:13.133939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:13.134727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:13.135200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:13.136032 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] RBAC: Adding query filter params (name=image) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:13.136481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f059-840e-4bd0-a021-1a6ea4884c61 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:13.136775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:13.139417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:13.150212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 368/727] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:13.152466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:13.176474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bd9f0-8f49-4bb3-802c-1802bf036c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:13.177263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bd9f0-8f49-4bb3-802c-1802bf036c11 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:13.177642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bd9f0-8f49-4bb3-802c-1802bf036c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:13.178055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bd9f0-8f49-4bb3-802c-1802bf036c11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:13.178839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bd9f0-8f49-4bb3-802c-1802bf036c11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:13.180431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bd9f0-8f49-4bb3-802c-1802bf036c11 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:13.262140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 360/728] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:51:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:13.269364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c294c-65b4-4852-b06c-2fbdc0584e6e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:13.286591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:13.318626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b81b6-899f-44cf-b89b-e0a3d175091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:13.319307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b81b6-899f-44cf-b89b-e0a3d175091f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:13.319589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b81b6-899f-44cf-b89b-e0a3d175091f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:13.319891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b81b6-899f-44cf-b89b-e0a3d175091f None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:13.320587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b81b6-899f-44cf-b89b-e0a3d175091f None admin] RBAC: Adding query filter params (type=image) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:13.320944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b81b6-899f-44cf-b89b-e0a3d175091f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:13.322401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b81b6-899f-44cf-b89b-e0a3d175091f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:13.328472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 369/729] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:13.873936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25fe9-5eb7-466e-bd11-93e384a57560 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:13.874354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25fe9-5eb7-466e-bd11-93e384a57560 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:13.874532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25fe9-5eb7-466e-bd11-93e384a57560 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:13.876109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 361/730] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:13.882180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c069a-10b2-43de-9d96-206744801bc9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:13.882468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c069a-10b2-43de-9d96-206744801bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:13.882743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c069a-10b2-43de-9d96-206744801bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:13.901734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1e2c069a-10b2-43de-9d96-206744801bc9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:13.902919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1e2c069a-10b2-43de-9d96-206744801bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:13.977041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 370/731] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:14.255490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25fe9-5eb7-466e-bd11-93e384a57560 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:14.298627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c13847-f549-45b3-baa3-505fb3b2ab67 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:14.299170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c13847-f549-45b3-baa3-505fb3b2ab67 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:14.299947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c13847-f549-45b3-baa3-505fb3b2ab67 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:14.301300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 371/732] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:14.306457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01bdff-3ec4-40ea-8a5c-ddbe818a9e2b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:14.306743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01bdff-3ec4-40ea-8a5c-ddbe818a9e2b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:14.307024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01bdff-3ec4-40ea-8a5c-ddbe818a9e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:14.326446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ac01bdff-3ec4-40ea-8a5c-ddbe818a9e2b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:14.327757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ac01bdff-3ec4-40ea-8a5c-ddbe818a9e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:14.373183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25fe9-5eb7-466e-bd11-93e384a57560 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:14.391036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:14.404894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 372/733] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:14.415691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:14.416377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:14.416653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:14.416921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:14.417497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:14.418855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:14.428488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:14.443050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:14.444840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 362/734] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:14.446494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01bdff-3ec4-40ea-8a5c-ddbe818a9e2b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:14.468767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:14.540207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01bdff-3ec4-40ea-8a5c-ddbe818a9e2b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:14.575857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:14.576466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:14.576741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:14.577007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:14.577625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:14.577924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:14.579274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:14.584867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 373/735] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:14.590607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:14.693353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:14.694884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5ab4-2584-483c-a651-e8a2b51e530a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:14.732436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:14.733153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:14.733433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:14.733702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:14.734322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:14.734618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:14.736011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:14.741468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 363/736] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:14.747530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:14.800178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:14.809860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:14.816130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89b616-449b-474a-89ff-1cbc1826fa96 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:14.856883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeba8d0-b59d-47e5-8ba9-a81745b61b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:14.857770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeba8d0-b59d-47e5-8ba9-a81745b61b6a None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:14.858047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeba8d0-b59d-47e5-8ba9-a81745b61b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:14.858419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeba8d0-b59d-47e5-8ba9-a81745b61b6a None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:14.859160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeba8d0-b59d-47e5-8ba9-a81745b61b6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:14.860530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeba8d0-b59d-47e5-8ba9-a81745b61b6a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:14.870272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc12b18-58c4-41a6-b918-f883fd8aa500 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:14.905787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:14.906386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:14.906664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:14.906932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:14.907640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] RBAC: Adding query filter params (interface=admin, service_id=130340b3022242d99dc8f19ee54be28b, region_id=RegionOne) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:14.907963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:14.910076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:14.915400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 364/737] 10.222.0.20 () {56 vars in 1249 bytes} [Wed Oct 26 10:51:14 2022] GET /identity/v3/endpoints?service_id=130340b3022242d99dc8f19ee54be28b&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:14.920907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:14.935110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 374/738] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:14 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:51:15.053583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300ee78-a9a7-4e5f-a46c-2380357a4784 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:15.062238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:15.104479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed532fb6-1060-45f6-a8ac-9a858b50c98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:15.105182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532fb6-1060-45f6-a8ac-9a858b50c98d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:15.105455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532fb6-1060-45f6-a8ac-9a858b50c98d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:15.105722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532fb6-1060-45f6-a8ac-9a858b50c98d None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:15.106315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed532fb6-1060-45f6-a8ac-9a858b50c98d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:15.107680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed532fb6-1060-45f6-a8ac-9a858b50c98d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:15.116199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 365/739] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:51:14 2022] GET /identity/v3/services => generated 1727 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 10:51:15.594478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b23f-8aa1-4282-81b0-d5a65e653cda None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:15.595326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b23f-8aa1-4282-81b0-d5a65e653cda None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:15.595746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b23f-8aa1-4282-81b0-d5a65e653cda None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:15.597760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 375/740] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:15.603528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d230c70-9b95-4409-8274-7453697937c8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:15.604410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d230c70-9b95-4409-8274-7453697937c8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:15.604915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d230c70-9b95-4409-8274-7453697937c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:15.622098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9214a-b9b3-4b5a-9e6e-4fc839390abf None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:15.622402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9214a-b9b3-4b5a-9e6e-4fc839390abf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:15.622646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9214a-b9b3-4b5a-9e6e-4fc839390abf None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:15.624247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 376/741] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:51:15.624731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6d230c70-9b95-4409-8274-7453697937c8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:15.625998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6d230c70-9b95-4409-8274-7453697937c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:15.629527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008023a-1fc3-4dec-a327-03eff8fb4da8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:15.629808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008023a-1fc3-4dec-a327-03eff8fb4da8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:15.630080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008023a-1fc3-4dec-a327-03eff8fb4da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:15.649552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1008023a-1fc3-4dec-a327-03eff8fb4da8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:15.650746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1008023a-1fc3-4dec-a327-03eff8fb4da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:15.709082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 366/742] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:15.727945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 377/743] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:15.977799 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d230c70-9b95-4409-8274-7453697937c8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:16.000463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008023a-1fc3-4dec-a327-03eff8fb4da8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:16.091558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:16.098254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d230c70-9b95-4409-8274-7453697937c8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:16.113734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:16.135571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008023a-1fc3-4dec-a327-03eff8fb4da8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:16.139710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:16.140367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:16.140645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:16.140917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:16.141581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:16.143300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:16.172461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:16.173384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:16.173841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:16.174366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:16.175205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:16.176889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:16.179763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING keystone.server.flask.common [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 26 10:51:16.229609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.sql.core [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 26 10:51:16.229609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:51:16.229609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [parameters: {'id': '60ec0e4f00024ea2a06190ed81e39c97', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 26 10:51:16.229609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81607) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60ec0e4f00024ea2a06190ed81e39c97.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60ec0e4f00024ea2a06190ed81e39c97. Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:51:16.237488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 26 10:51:16.240649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application yield resource Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise self.value Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application session.commit() Oct 26 10:51:16.241642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.flush() Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application compat.raise_( Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 10:51:16.242554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 10:51:16.244360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:51:16.246342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [parameters: {'id': '60ec0e4f00024ea2a06190ed81e39c97', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:16.248522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:16.249661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60ec0e4f00024ea2a06190ed81e39c97. Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 378/744] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Oct 26 10:51:15 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:16.251332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:16.272104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 367/745] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:15 2022] POST /identity/v3/services => generated 242 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:16.373141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2170d-dc64-4379-996c-ebab8a7225ca None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:16.422532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357083a-4833-4448-b0ab-6a25d0338f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:16.423185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357083a-4833-4448-b0ab-6a25d0338f1b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:16.423459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357083a-4833-4448-b0ab-6a25d0338f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:16.423724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357083a-4833-4448-b0ab-6a25d0338f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:16.432068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357083a-4833-4448-b0ab-6a25d0338f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:16.433778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357083a-4833-4448-b0ab-6a25d0338f1b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:16.435774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8357083a-4833-4448-b0ab-6a25d0338f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:51:16.446502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:16.453708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:16.454628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 379/746] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:16.455126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:16.563183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55208b2e-79e8-4574-86c3-10857b5e82d8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:16.598242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa55f20-af9c-4f5b-b774-e8b88f42a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:16.598858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa55f20-af9c-4f5b-b774-e8b88f42a622 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:16.599134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa55f20-af9c-4f5b-b774-e8b88f42a622 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:16.599404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa55f20-af9c-4f5b-b774-e8b88f42a622 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:16.600097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa55f20-af9c-4f5b-b774-e8b88f42a622 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:16.600398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa55f20-af9c-4f5b-b774-e8b88f42a622 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:16.601832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa55f20-af9c-4f5b-b774-e8b88f42a622 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:16.619837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 368/747] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:51:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:17.240969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4213633-1996-4e73-b3fc-01f9a98026da None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:17.241487 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4213633-1996-4e73-b3fc-01f9a98026da None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:17.241487 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4213633-1996-4e73-b3fc-01f9a98026da None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:17.243070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 380/748] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:17.248513 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f710d-519e-418a-bd57-6ed07b0361d2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:17.248793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f710d-519e-418a-bd57-6ed07b0361d2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:17.249063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f710d-519e-418a-bd57-6ed07b0361d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:17.272360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-694f710d-519e-418a-bd57-6ed07b0361d2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:17.273637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-694f710d-519e-418a-bd57-6ed07b0361d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:17.288512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:17.344637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 369/749] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:17.386775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4e10d-f965-4846-86d7-d9601de620d9 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:17.387351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4e10d-f965-4846-86d7-d9601de620d9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:17.387839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4e10d-f965-4846-86d7-d9601de620d9 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:17.389637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 381/750] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:17.395205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a82a7f-1c23-4778-9292-9f375f42b02a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:17.395491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a82a7f-1c23-4778-9292-9f375f42b02a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:17.395760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a82a7f-1c23-4778-9292-9f375f42b02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:17.414560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-26a82a7f-1c23-4778-9292-9f375f42b02a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:17.415729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-26a82a7f-1c23-4778-9292-9f375f42b02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:17.491170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 370/751] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:17.631589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4e10d-f965-4846-86d7-d9601de620d9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:17.741799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4e10d-f965-4846-86d7-d9601de620d9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:17.767157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a82a7f-1c23-4778-9292-9f375f42b02a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:17.778672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:17.779302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:17.779575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:17.779868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:17.780445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:17.781804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:17.790948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:17.791952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:17.792816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 382/752] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:17.801388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:17.868857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:17.885415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:17.897912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a82a7f-1c23-4778-9292-9f375f42b02a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:17.932317 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbc6ce-501e-4818-b406-c71bcdef9ab1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:17.943188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:17.944395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:17.944767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:17.945244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:17.945841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:17.948651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:17.962622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:17.963645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:17.969797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 371/753] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 26 10:51:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:17.970600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:17.988606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:17.989281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:17.989569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:17.989840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:17.990944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:17.991390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:17.992819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:17.998436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 383/754] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:51:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:18.004675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:18.105672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7976c5-837e-4e3f-8e04-daa75334a352 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:18.129903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05a621-85d4-4d62-bcdf-08c0727d91f2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:18.142442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:18.144003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.144522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.144995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.145822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:18.146328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:18.147952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:18.158710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 372/755] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:18.160009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:18.160553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:18.164058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:18.171553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:18.172205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.180411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.180411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.180411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:18.180411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:18.180411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:18.183870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 384/756] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:51:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:18.188301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:18.279432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e74cd-2a76-4d56-ab33-d88f87c6c5e6 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:18.314446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ed141-8dd0-4ced-8e45-69c9c385058e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:18.317303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:18.318094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.318405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.318780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.320139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:18.320139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:18.321478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:18.331896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 373/757] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 26 10:51:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:18.332957 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:18.364523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:18.365143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.365426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.365704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.366687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] RBAC: Adding query filter params (interface=public, service_id=304de1832ae0406bacfbf6d5e186d509, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:18.367005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:18.368535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:18.380112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 385/758] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:51:18 2022] GET /identity/v3/endpoints?service_id=304de1832ae0406bacfbf6d5e186d509&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:18.382144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:18.424315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:18.455078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:18.474082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f191a-6cef-4204-93d8-b71c03e45c3a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:18.517440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf003-fb8a-4cc7-b880-0f55413ebb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:18.518545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf003-fb8a-4cc7-b880-0f55413ebb7b None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.519031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf003-fb8a-4cc7-b880-0f55413ebb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.519534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf003-fb8a-4cc7-b880-0f55413ebb7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.520462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf003-fb8a-4cc7-b880-0f55413ebb7b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:18.520880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca069b3-82ef-45ce-bf5d-74dbfe27c2b7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:18.522481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf003-fb8a-4cc7-b880-0f55413ebb7b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:18.557740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9af033-caa4-4f90-bf71-11333f23bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:18.558365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9af033-caa4-4f90-bf71-11333f23bce7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.558648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9af033-caa4-4f90-bf71-11333f23bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.558922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9af033-caa4-4f90-bf71-11333f23bce7 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.559516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9af033-caa4-4f90-bf71-11333f23bce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:18.560914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9af033-caa4-4f90-bf71-11333f23bce7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:18.571886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 386/759] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:51:18 2022] GET /identity/v3/services => generated 1957 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 10:51:18.588660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 374/760] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:51:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:18.604426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d5750-c6e6-4881-b008-2a4514c7e88a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.604713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d5750-c6e6-4881-b008-2a4514c7e88a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.604991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d5750-c6e6-4881-b008-2a4514c7e88a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.606754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 387/761] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:18.613475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9ca9-95a1-4cd4-90f8-3103655008c8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.613760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9ca9-95a1-4cd4-90f8-3103655008c8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.614040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc9ca9-95a1-4cd4-90f8-3103655008c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.635646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2cfc9ca9-95a1-4cd4-90f8-3103655008c8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:18.637370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2cfc9ca9-95a1-4cd4-90f8-3103655008c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:18.730810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 375/762] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:18 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:18.938298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eb58c-c38e-4900-b657-7b507da55b47 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.938560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eb58c-c38e-4900-b657-7b507da55b47 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.938878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eb58c-c38e-4900-b657-7b507da55b47 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.941070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 388/763] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:18.949452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9b27b-d7f6-426d-a2ef-15e976a464f8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:18.949930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9b27b-d7f6-426d-a2ef-15e976a464f8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:18.950201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9b27b-d7f6-426d-a2ef-15e976a464f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:18.970791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e0a9b27b-d7f6-426d-a2ef-15e976a464f8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:18.972161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e0a9b27b-d7f6-426d-a2ef-15e976a464f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:19.015931 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eb58c-c38e-4900-b657-7b507da55b47 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.026341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:19.047036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 376/764] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:18 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:19.145555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eb58c-c38e-4900-b657-7b507da55b47 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.164929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:19.182370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.183004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.183342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.183620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] PATH_INFO: `/v3/services/image` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.184269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.185646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:19.195899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:19.196888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.197731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 389/765] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Oct 26 10:51:19 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:19.203426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9b27b-d7f6-426d-a2ef-15e976a464f8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.322725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9b27b-d7f6-426d-a2ef-15e976a464f8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.343472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.345200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.345915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.346391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.346391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.347524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:19.347846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.350109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.356641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 377/766] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:19.363177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.429742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:19.457804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf9911-bdae-4d68-addf-4cbbfcd3098b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.494415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.494966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.495243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.495511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.496428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.497602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.499520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c5aed-e9ff-48f1-93e2-8ab3d1c7f6a7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.537228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.537883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.538165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.538437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.539055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] RBAC: Adding query filter params (type=image) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:19.539355 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.540747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.545772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 378/767] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:19.551343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.557099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.sql.core [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 26 10:51:19.557099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 26 10:51:19.557099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [parameters: {'user_id': 'a931dd09bbd4471981738588ef116c77', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 26 10:51:19.557099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81607) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:51:19.564162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:51:19.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application yield resource Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise self.value Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application session.commit() Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 10:51:19.566240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.flush() Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application compat.raise_( Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 10:51:19.567150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:51:19.569096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:51:19.571138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a931dd09bbd4471981738588ef116c77', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:19.573321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 26 10:51:19.577021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 390/768] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 26 10:51:19 2022] POST /identity/v3/users => generated 160 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:19.579369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.632738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:19.695153 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82491ab-34b8-4eaa-ae36-6695738279a3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.710078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603b0e7-2b87-4e98-babf-dec6921235fa None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.725788 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:19.735542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.738316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.738316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.738316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.747050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.748593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.749201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.749543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.750020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.750442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.751282 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] RBAC: Adding query filter params (interface=public, service_id=dd74dcb3d7494fde884565bbdc4f37af, region_id=RegionOne) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:19.751918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.754056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:51:19.764177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:19.765357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:19.766573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 391/769] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:51:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:19.768360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.769119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 379/770] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:51:19 2022] GET /identity/v3/endpoints?service_id=dd74dcb3d7494fde884565bbdc4f37af&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:19.775087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:19.888495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4362-1f49-4c5a-8385-ef86ccb47441 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.892567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:19.898079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b802-f6ad-4278-a095-98d76feff153 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:19.930310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bde39-8e3d-4362-9f18-197f895be4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.931327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bde39-8e3d-4362-9f18-197f895be4ab None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.931755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bde39-8e3d-4362-9f18-197f895be4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.932550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bde39-8e3d-4362-9f18-197f895be4ab None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.933583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bde39-8e3d-4362-9f18-197f895be4ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.935119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bde39-8e3d-4362-9f18-197f895be4ab None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.941516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0474d5-33a1-4ef4-8ab5-8038aff5274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:19.942254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0474d5-33a1-4ef4-8ab5-8038aff5274c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:19.942623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0474d5-33a1-4ef4-8ab5-8038aff5274c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:19.942988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0474d5-33a1-4ef4-8ab5-8038aff5274c None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:19.943774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0474d5-33a1-4ef4-8ab5-8038aff5274c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:19.944189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0474d5-33a1-4ef4-8ab5-8038aff5274c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:19.945705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0474d5-33a1-4ef4-8ab5-8038aff5274c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:19.951959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 392/771] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:51:19 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 10:51:19.991398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 380/772] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 26 10:51:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:20.895461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1fc92-12a1-4a0a-adad-da24d9e29963 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:20.896058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1fc92-12a1-4a0a-adad-da24d9e29963 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:20.896058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1fc92-12a1-4a0a-adad-da24d9e29963 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:20.897637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 393/773] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:20.903527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3456dc-491f-4beb-a420-e00f732b7b3f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:20.903886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3456dc-491f-4beb-a420-e00f732b7b3f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:20.904118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3456dc-491f-4beb-a420-e00f732b7b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:20.923189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4b3456dc-491f-4beb-a420-e00f732b7b3f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:20.924439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4b3456dc-491f-4beb-a420-e00f732b7b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:20.940796 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83588cd6-2b26-45d8-b3a4-ab375f087867 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:20.941084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83588cd6-2b26-45d8-b3a4-ab375f087867 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:20.941361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83588cd6-2b26-45d8-b3a4-ab375f087867 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:20.942775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 394/774] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:20.947882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c70068-2ac1-42b3-9091-6f142b238e13 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:20.948173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c70068-2ac1-42b3-9091-6f142b238e13 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:20.948451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c70068-2ac1-42b3-9091-6f142b238e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:20.954174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:20.967294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d8c70068-2ac1-42b3-9091-6f142b238e13 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:20.968849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d8c70068-2ac1-42b3-9091-6f142b238e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:20.995550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 381/775] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:21.050061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 395/776] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:21.267117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3456dc-491f-4beb-a420-e00f732b7b3f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:21.321233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c70068-2ac1-42b3-9091-6f142b238e13 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:21.369918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3456dc-491f-4beb-a420-e00f732b7b3f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:21.405758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:21.406394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:21.406674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:21.406948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:21.407531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:21.409573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:21.418340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:21.419352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 10:51:21.420252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:21.425208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 382/777] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:21.425708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:21.426334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:21.431020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c70068-2ac1-42b3-9091-6f142b238e13 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:21.457181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:21.469294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:21.469941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:21.470269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:21.470543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:21.472547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:21.474617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:21.479820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 396/778] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:21.481875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:21.537596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c526e4-d0dd-4cd2-8c02-e731b02e112a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:21.573048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:21.573801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:21.573932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:21.574195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:21.574820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:21.575146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:21.576562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:21.587155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 383/779] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:51:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:21.588352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:21.589377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ff143-dbaf-4292-8d7f-2374641cc053 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:21.624789 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:21.625393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:21.625670 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:21.625941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:21.626582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:21.628620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:21.689165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:21.698785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19e06c-8d9c-467d-b0c7-74ab723be2e2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:21.714143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 397/780] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:21 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:21.737324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee689d-5460-43b5-bd52-f774559c24f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:21.738046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee689d-5460-43b5-bd52-f774559c24f1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:21.738309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee689d-5460-43b5-bd52-f774559c24f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:21.738587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee689d-5460-43b5-bd52-f774559c24f1 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:21.739234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee689d-5460-43b5-bd52-f774559c24f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:21.739541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee689d-5460-43b5-bd52-f774559c24f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:21.741025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee689d-5460-43b5-bd52-f774559c24f1 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:21.746978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 384/781] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 26 10:51:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:21.753445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:21.775510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:21.865913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c836a8-1e8d-470c-83f2-4ad005958b9a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:21.904230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27535cd6-ca60-4bb8-8229-e37e7297e0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:21.905568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27535cd6-ca60-4bb8-8229-e37e7297e0c7 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:21.905852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27535cd6-ca60-4bb8-8229-e37e7297e0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:21.906127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27535cd6-ca60-4bb8-8229-e37e7297e0c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:21.906793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27535cd6-ca60-4bb8-8229-e37e7297e0c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:21.908759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27535cd6-ca60-4bb8-8229-e37e7297e0c7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:22.015692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 398/782] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:51:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:22.240983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4ab90-2739-49ca-8f6a-7e4a8a4ade4d None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:22.241496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4ab90-2739-49ca-8f6a-7e4a8a4ade4d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:22.241496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4ab90-2739-49ca-8f6a-7e4a8a4ade4d None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:22.243033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 385/783] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:22.248565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37064830-f002-4d71-a674-980444531be9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:22.248922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37064830-f002-4d71-a674-980444531be9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:22.249170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37064830-f002-4d71-a674-980444531be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:22.268443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-37064830-f002-4d71-a674-980444531be9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:22.269750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-37064830-f002-4d71-a674-980444531be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:22.341223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:22.346934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 399/784] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:22.614528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4ab90-2739-49ca-8f6a-7e4a8a4ade4d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:22.721287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4ab90-2739-49ca-8f6a-7e4a8a4ade4d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:22.755870 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:22.756523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:22.756806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:22.757076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] PATH_INFO: `/v3/services/image` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:22.757645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:22.759324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:22.768191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:22.769168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 10:51:22.770028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:22.774297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 386/785] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Oct 26 10:51:22 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:22.775456 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37064830-f002-4d71-a674-980444531be9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:22.877557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37064830-f002-4d71-a674-980444531be9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:22.911340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:22.912042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:22.912252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:22.912527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:22.913138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] RBAC: Adding query filter params (name=image) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:22.913440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:22.914801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:22.925133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 400/786] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:22.926111 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:23.065916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:23.143677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc67ffd-b90d-4da5-b622-570d8c61cd4a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:23.177673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb619163-14ac-47ea-9259-ed763797b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:23.178274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619163-14ac-47ea-9259-ed763797b567 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:23.178550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619163-14ac-47ea-9259-ed763797b567 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:23.178823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619163-14ac-47ea-9259-ed763797b567 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:23.179437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb619163-14ac-47ea-9259-ed763797b567 None admin] RBAC: Adding query filter params (type=image) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:23.179735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb619163-14ac-47ea-9259-ed763797b567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:23.181143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb619163-14ac-47ea-9259-ed763797b567 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:23.186172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 387/787] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:23.191760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:23.253143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:23.291444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbf985-3339-40b8-9ff5-d5a9705092cc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:23.324661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2376-8e89-4d1a-a731-886fdfb2bf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:23.325253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2376-8e89-4d1a-a731-886fdfb2bf13 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:23.325531 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2376-8e89-4d1a-a731-886fdfb2bf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:23.325800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2376-8e89-4d1a-a731-886fdfb2bf13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:23.326455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f2376-8e89-4d1a-a731-886fdfb2bf13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:23.327883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f2376-8e89-4d1a-a731-886fdfb2bf13 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:23.410644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 401/788] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:51:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:23.949961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ad84a-eebd-4e25-98c7-ce470959f36c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:23.950505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ad84a-eebd-4e25-98c7-ce470959f36c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:23.950505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ad84a-eebd-4e25-98c7-ce470959f36c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:23.952156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 388/789] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:23.957547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5cde-368f-4fd3-aa84-0fb343cb2050 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:23.958020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5cde-368f-4fd3-aa84-0fb343cb2050 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:23.958100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5cde-368f-4fd3-aa84-0fb343cb2050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:23.976450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-94cf5cde-368f-4fd3-aa84-0fb343cb2050 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:23.977637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-94cf5cde-368f-4fd3-aa84-0fb343cb2050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:24.056052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 402/790] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:24.327924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ad84a-eebd-4e25-98c7-ce470959f36c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:24.339984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d36155-8c3b-44de-9022-22404b168dac None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:24.340287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d36155-8c3b-44de-9022-22404b168dac None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:24.340868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d36155-8c3b-44de-9022-22404b168dac None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:24.342233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 403/791] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:51:24.348402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034363c-fe26-48b1-a2bc-1557dfb4bca0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:24.348690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034363c-fe26-48b1-a2bc-1557dfb4bca0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:24.348968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034363c-fe26-48b1-a2bc-1557dfb4bca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:24.369272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0034363c-fe26-48b1-a2bc-1557dfb4bca0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:24.370735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0034363c-fe26-48b1-a2bc-1557dfb4bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:24.418405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:24.419547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:24.436936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ad84a-eebd-4e25-98c7-ce470959f36c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:24.447823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 404/792] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:24.474019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:24.474635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:24.474913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:24.475181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:24.476314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:24.477728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:24.479835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 389/793] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:24.484868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034363c-fe26-48b1-a2bc-1557dfb4bca0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:24.583721 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034363c-fe26-48b1-a2bc-1557dfb4bca0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:24.618267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:24.618877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:24.619155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:24.619427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:24.620114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:24.621539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:24.704367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 405/794] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:51:24 2022] POST /identity/v3/users => generated 285 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:24.734566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:24.846775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bcadc-456d-4828-ad12-0e5ab93825d9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:24.887828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:24.888262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:24.888412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:24.888669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:24.899861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:24.905493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:24.906436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:24.907298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 390/795] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 26 10:51:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:24.913589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:24.947737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:25.015939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e7d51-b590-49ab-b4e7-9bfa36a68d1f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:25.051691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:25.052407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.052692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.052964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.053627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:25.053929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:25.055287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:25.064052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 406/796] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 26 10:51:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:25.070010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:25.086429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:25.171963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14331f7-1fec-43be-9e46-b2ab9a914d87 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:25.206072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:25.206699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.206977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.207270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.217451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:25.218907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:25.235667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:25.236944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:25.237908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:25.237908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:25.237908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:25.240126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 391/797] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:25.246414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:25.341072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:25.347959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214682fa-1f50-455c-a74e-5de18e277d2f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:25.381961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:25.382560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.382833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.383100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.383722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:25.384311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:25.385653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:25.401874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 407/798] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:25.403336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:25.469237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:25.514029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fcd28-1b1c-4e8f-b3c6-bf02bd1e4da8 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:25.547481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8eeb5f-82aa-4c82-800e-a175f3aabc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:25.548140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8eeb5f-82aa-4c82-800e-a175f3aabc55 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.548418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8eeb5f-82aa-4c82-800e-a175f3aabc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.549220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8eeb5f-82aa-4c82-800e-a175f3aabc55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.556883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8eeb5f-82aa-4c82-800e-a175f3aabc55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:25.558332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8eeb5f-82aa-4c82-800e-a175f3aabc55 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8c8eeb5f-82aa-4c82-800e-a175f3aabc55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:51:25.570285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:25.571232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:51:25.572583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:25.576090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 392/799] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 26 10:51:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:25.577302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:25.597560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ddb1c-19c6-48e2-bc38-0566bf402bf1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.597771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ddb1c-19c6-48e2-bc38-0566bf402bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.597865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ddb1c-19c6-48e2-bc38-0566bf402bf1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.599430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 393/800] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:51:25.604244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f1c0-5bad-4310-99d1-109d302b75ba None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.604528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f1c0-5bad-4310-99d1-109d302b75ba None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.604803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86f1c0-5bad-4310-99d1-109d302b75ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.623659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1f86f1c0-5bad-4310-99d1-109d302b75ba None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:25.624874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1f86f1c0-5bad-4310-99d1-109d302b75ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:25.692600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1d69c-ce52-441e-aa6a-5f21ed2c78fa None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:25.699376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 394/801] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:25.728889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:25.729404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.729676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.729941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.730738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:25.731042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:25.732468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:25.755688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 408/802] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 26 10:51:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:25.756905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86f1c0-5bad-4310-99d1-109d302b75ba None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:25.813852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:25.859288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86f1c0-5bad-4310-99d1-109d302b75ba None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:25.894580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:25.895197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:25.895470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:25.895734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:25.905507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:25.906924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:25.922038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:25.922996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:25.928700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:25.928700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:25.928700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:25.928700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 395/803] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:25.929387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:25.944913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:25.969801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:26.038949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c59ac-a85a-49f5-b6a4-cfefbf3e1694 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:26.076731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:26.077074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30480d-85b6-40d6-a4d2-c73553902604 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:26.077501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.077586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.077860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.078495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:26.078794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:26.080843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:26.085445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:26.092074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 409/804] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:26.098881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:26.122223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:26.122744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.123024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.123293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.124109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:26.125589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:26.170462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:26.173374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.sql.core [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:51:26.173374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:51:26.173374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [parameters: {'id': '5bed2a24e69e485aa950c9fbebdb1c0c', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 10:51:26.173374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81608) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:51:26.190028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result.read() Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 26 10:51:26.190897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application yield resource Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise self.value Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application session.commit() Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 10:51:26.191894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.session.flush() Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application compat.raise_( Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 10:51:26.192806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 10:51:26.193690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application result.read() Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:51:26.194557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [parameters: {'id': '5bed2a24e69e485aa950c9fbebdb1c0c', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 26 10:51:26.195467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:26.197847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 396/805] 10.222.0.20 () {60 vars in 1137 bytes} [Wed Oct 26 10:51:25 2022] POST /identity/v3/domains => generated 190 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:26.199005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:26.212755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77267afe-bcb2-4d7d-9d23-90e6a43211d8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:26.251915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:26.252730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.253044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.253679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.263055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:26.264979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:51:26.280440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:26.281363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:26.282332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 410/806] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:26.288467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:26.318664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba6e63-55b6-45fb-bd21-c5f5c48d4d95 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:26.365640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:26.366276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.366550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.366813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.378526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:26.380003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:26.397111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4911677-6da8-4aeb-adae-0df2c0634556 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:26.399123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:26.400039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:26.400926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:26.400926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:26.400926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:26.401094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 397/807] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:26.407038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:26.433154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:26.433338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:26.440346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:26.441166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.441585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.442012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.443012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:26.443828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:26.445681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:26.459869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 411/808] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:26.465753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:26.516430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be2164-a2b7-4a65-822b-e6c199961b1a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:26.554053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87868bf7-0659-41e6-8117-294fd95f519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:26.554737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868bf7-0659-41e6-8117-294fd95f519e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.555015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868bf7-0659-41e6-8117-294fd95f519e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.555339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868bf7-0659-41e6-8117-294fd95f519e None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.556053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87868bf7-0659-41e6-8117-294fd95f519e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:26.556455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87868bf7-0659-41e6-8117-294fd95f519e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:26.557836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87868bf7-0659-41e6-8117-294fd95f519e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:26.570429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 398/809] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:26.577923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d021-86d8-4c59-8c34-67e14f943d24 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:26.618322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70320dde-bb41-4f05-9b8a-e7954a9c09d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:26.619312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320dde-bb41-4f05-9b8a-e7954a9c09d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.619847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320dde-bb41-4f05-9b8a-e7954a9c09d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.620315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320dde-bb41-4f05-9b8a-e7954a9c09d0 None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/c3d05bdc4c9b4f24b1d58ef157b32ae0/roles/780ae8b185504927a727c332e2260a6d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.622866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70320dde-bb41-4f05-9b8a-e7954a9c09d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=c3d05bdc4c9b4f24b1d58ef157b32ae0, role_id=780ae8b185504927a727c332e2260a6d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:26.624592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70320dde-bb41-4f05-9b8a-e7954a9c09d0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:26.693289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 412/810] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:51:26 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/c3d05bdc4c9b4f24b1d58ef157b32ae0/roles/780ae8b185504927a727c332e2260a6d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:51:26.953685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e830659-577d-43ec-bb89-8ae4168e80e7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.953970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e830659-577d-43ec-bb89-8ae4168e80e7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.954249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e830659-577d-43ec-bb89-8ae4168e80e7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.956182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 399/811] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:26.961657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8ea24-c828-4d65-a72c-f84eeeec58e2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:26.962024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8ea24-c828-4d65-a72c-f84eeeec58e2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:26.962364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8ea24-c828-4d65-a72c-f84eeeec58e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:26.981494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1fe8ea24-c828-4d65-a72c-f84eeeec58e2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:26.982657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1fe8ea24-c828-4d65-a72c-f84eeeec58e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:27.029290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:27.050680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 413/812] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:27.327165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e830659-577d-43ec-bb89-8ae4168e80e7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:27.411096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:27.430531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e830659-577d-43ec-bb89-8ae4168e80e7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:27.464417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:27.465036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:27.465309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:27.465574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:27.476229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:27.480937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:27.482023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:27.482879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 400/813] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:51:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:27.488347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8ea24-c828-4d65-a72c-f84eeeec58e2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:27.590289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8ea24-c828-4d65-a72c-f84eeeec58e2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:27.623672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:27.624292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:27.624563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:27.624827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:27.625471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:27.625765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:27.627524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:27.635690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 414/814] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:51:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:27.641456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:27.740772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d30e0a-9790-431e-b831-0f327c53d338 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:27.774051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:27.774669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:27.774949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:27.775222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] PATH_INFO: `/v3/users/7679d070532c47fd97baee2e7f782dea` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:27.787180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:27.789535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] RBAC: Authorizing `identity:get_user(user_id=7679d070532c47fd97baee2e7f782dea)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:27.790969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:27.792586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 401/815] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:27 2022] GET /identity/v3/users/7679d070532c47fd97baee2e7f782dea => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:27.798454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:27.844935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:27.900346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc00e3-96c0-4aa3-bb52-5c5ed65fbf9c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:27.933047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab741a4e-f727-4e7c-bb5a-742eb1bec477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:27.933670 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab741a4e-f727-4e7c-bb5a-742eb1bec477 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:27.933945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab741a4e-f727-4e7c-bb5a-742eb1bec477 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:27.934213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab741a4e-f727-4e7c-bb5a-742eb1bec477 None admin] PATH_INFO: `/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:27.935301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab741a4e-f727-4e7c-bb5a-742eb1bec477 None admin] RBAC: Authorizing `identity:get_project(project_id=b8d440e8301d40a78a6bb6029b17b2eb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:27.936737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab741a4e-f727-4e7c-bb5a-742eb1bec477 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:27.938354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 415/816] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:51:27 2022] GET /identity/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:27.945391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:28.043006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd53470-5256-442f-a443-2c76ff4c243a None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:28.076755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849725f-ef9e-4c2e-98c1-5f21333705e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:28.077391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849725f-ef9e-4c2e-98c1-5f21333705e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:28.077683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849725f-ef9e-4c2e-98c1-5f21333705e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:28.077947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849725f-ef9e-4c2e-98c1-5f21333705e5 None admin] PATH_INFO: `/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb/users/7679d070532c47fd97baee2e7f782dea/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:28.080929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849725f-ef9e-4c2e-98c1-5f21333705e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8d440e8301d40a78a6bb6029b17b2eb, user_id=7679d070532c47fd97baee2e7f782dea, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:28.082341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849725f-ef9e-4c2e-98c1-5f21333705e5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:28.145554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 402/817] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:51:27 2022] PUT /identity/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb/users/7679d070532c47fd97baee2e7f782dea/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:51:28.847631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa357fe7-2f58-464c-9ef2-6a765ddf675f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:28.848226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa357fe7-2f58-464c-9ef2-6a765ddf675f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:28.848226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa357fe7-2f58-464c-9ef2-6a765ddf675f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:28.849563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 416/818] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:28.854989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdfee-cd04-4a22-ad2e-262fa1d5d2d8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:28.855427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdfee-cd04-4a22-ad2e-262fa1d5d2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:28.855528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdfee-cd04-4a22-ad2e-262fa1d5d2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:28.873456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c8fcdfee-cd04-4a22-ad2e-262fa1d5d2d8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:28.874652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c8fcdfee-cd04-4a22-ad2e-262fa1d5d2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:28.929006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:28.940570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 403/819] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:29.207264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa357fe7-2f58-464c-9ef2-6a765ddf675f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:29.308248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa357fe7-2f58-464c-9ef2-6a765ddf675f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:29.321134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:29.342116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:29.342751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:29.343026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:29.343294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:29.354331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:29.359516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:29.360510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:29.361460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 417/820] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 26 10:51:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:29.366803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdfee-cd04-4a22-ad2e-262fa1d5d2d8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:29.463933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdfee-cd04-4a22-ad2e-262fa1d5d2d8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:29.496156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:29.496753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:29.497052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:29.497325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:29.497977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:29.498279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:29.499642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:29.508087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 404/821] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 26 10:51:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:29.514289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:29.611224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072533b6-eade-44d2-a888-43009b59f8fc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:29.643778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:29.644409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:29.644682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:29.644946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:29.654912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:29.656357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:29.670297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:29.671449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:29.672819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:29.672819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:29.672819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:29.672819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 418/822] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:29.677916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:29.730179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:29.777552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0985e0-7040-4f1c-bc4d-8994f6c6c478 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:29.811467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:29.812105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:29.812385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:29.812654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:29.813279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:29.813579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:29.815579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:29.827068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 405/823] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:29.832679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:29.858092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:29.932545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70299d95-7782-424e-b3a1-0c1bb32d2452 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:29.965921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:29.966541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:29.966819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:29.967092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:29.975369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:29.976869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:51:29.988522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:29.989604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:29.990525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 419/824] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 26 10:51:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:29.995322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.094200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1d6ca-db21-4b91-8aec-9365cf1d0ad3 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:30.106982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:30.128087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f900adb-e2a5-4ba2-82f9-0370ecfbe877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:30.128683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f900adb-e2a5-4ba2-82f9-0370ecfbe877 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.128960 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f900adb-e2a5-4ba2-82f9-0370ecfbe877 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.129229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f900adb-e2a5-4ba2-82f9-0370ecfbe877 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.129939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f900adb-e2a5-4ba2-82f9-0370ecfbe877 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:30.130242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f900adb-e2a5-4ba2-82f9-0370ecfbe877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:30.131657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f900adb-e2a5-4ba2-82f9-0370ecfbe877 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:30.149525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 406/825] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 26 10:51:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:30.155224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.249465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:30.264924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb7528-eb1f-4a88-8280-c37e3541e299 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:30.287342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bc5c5-4595-412b-b17f-01778851c190 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.287628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bc5c5-4595-412b-b17f-01778851c190 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.287935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bc5c5-4595-412b-b17f-01778851c190 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.289265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 407/826] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:51:30.294309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505a6d-3b2a-490a-919c-460a4021af1a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.294590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505a6d-3b2a-490a-919c-460a4021af1a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.294862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505a6d-3b2a-490a-919c-460a4021af1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.298313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:30.299519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.299725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.300088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.312923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3b505a6d-3b2a-490a-919c-460a4021af1a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:30.314478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3b505a6d-3b2a-490a-919c-460a4021af1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:30.368846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:30.370779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:30.380711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 408/827] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 10:51:30.450016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:30.470573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 420/828] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:51:30.472223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b505a6d-3b2a-490a-919c-460a4021af1a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.565694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:30.568486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b505a6d-3b2a-490a-919c-460a4021af1a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:30.602097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:30.602714 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.602991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.603260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.612999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:30.614438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:51:30.628659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:30.629780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:51:30.630874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:51:30.630874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:51:30.630874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:30.630874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 409/829] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:30.635647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.650399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.733086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5cb6-830e-4382-95d8-a9b810da9e7d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:30.747718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a23ea8-d15d-4e1e-a24f-3b80f9aa62ee None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:30.767450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:30.768180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.768451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.768715 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.769341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:30.769640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:30.770988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:30.781835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 421/830] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:30.781973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:30.782460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.783567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.783714 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.787979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.794645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:30.799507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:30.800645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:30.801499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 410/831] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:30.806412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.821253 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:30.836569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:30.890144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e09343-e129-4844-82b2-fdbcccae32d5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:30.905185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766db23-3e3d-4460-b9df-d8595a63cba7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:30.924419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:30.925042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.925316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.925585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.932789 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:30.934221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:30.938550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:30.939810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:30.940117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:30.940385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:30.941034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:30.941333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:30.942688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:51:30.946047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:30.946897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:51:30.947826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:30.955647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 411/832] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:30.955833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 422/833] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:51:30.955927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:30.956846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:31.028558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:31.056357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbcad6-2c3e-47b6-8701-c3a2d185d990 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:31.058464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acbaf7-4918-4489-80bd-4bcf89638d4b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:31.060782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:31.090184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:31.090846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:31.091126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:31.091400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:31.092127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:31.092363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:31.092483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:31.092988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:31.093262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:31.093528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] PATH_INFO: `/v3/users/7679d070532c47fd97baee2e7f782dea` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:31.093837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:31.094663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] RBAC: Authorizing `identity:get_user(user_id=7679d070532c47fd97baee2e7f782dea)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:31.096632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:31.098213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 423/834] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/users/7679d070532c47fd97baee2e7f782dea => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:31.103889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:31.112505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 412/835] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:31.113462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:31.203038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8fe26-696f-48bd-b449-37b9dd8b145a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:31.213258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e9b01-fb0e-4a94-8277-fa39f054b78f None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:31.237586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:31.238292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:31.238640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:31.238979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:31.246755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:31.247346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:31.247622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:31.247926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:31.248643 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] RBAC: Adding query filter params (role.id=780ae8b185504927a727c332e2260a6d, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=c3d05bdc4c9b4f24b1d58ef157b32ae0) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:31.248947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:31.250365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:31.252919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:31.257792 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 413/836] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:51:31 2022] GET /identity/v3/role_assignments?user.id=c3d05bdc4c9b4f24b1d58ef157b32ae0&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=780ae8b185504927a727c332e2260a6d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:31.283767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:31.312437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:31.314522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:31.418712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 424/837] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:51:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 26 10:51:31.426279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:31.528506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60375f-b353-44bb-9863-6d9a36bab2ec None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:31.562606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58bbc8-30ed-45a9-96e6-b2e7f9bbdc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:31.563237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58bbc8-30ed-45a9-96e6-b2e7f9bbdc82 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:31.563516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58bbc8-30ed-45a9-96e6-b2e7f9bbdc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:31.563868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58bbc8-30ed-45a9-96e6-b2e7f9bbdc82 None admin] PATH_INFO: `/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:31.564989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58bbc8-30ed-45a9-96e6-b2e7f9bbdc82 None admin] RBAC: Authorizing `identity:get_project(project_id=b8d440e8301d40a78a6bb6029b17b2eb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:31.566417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58bbc8-30ed-45a9-96e6-b2e7f9bbdc82 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:31.568220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 414/838] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:51:31 2022] GET /identity/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:31.574261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:31.645903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:31.676040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb213f8-2c68-406e-b969-51ac9cc32ae8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:31.711197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71c71b-f8ec-428f-9570-8d155d2c7ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:31.711831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c71b-f8ec-428f-9570-8d155d2c7ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:31.712118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c71b-f8ec-428f-9570-8d155d2c7ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:31.712390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c71b-f8ec-428f-9570-8d155d2c7ac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:31.713112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71c71b-f8ec-428f-9570-8d155d2c7ac5 None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=b8d440e8301d40a78a6bb6029b17b2eb, user.id=7679d070532c47fd97baee2e7f782dea) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:31.713414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71c71b-f8ec-428f-9570-8d155d2c7ac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:31.714845 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71c71b-f8ec-428f-9570-8d155d2c7ac5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:31.721574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 425/839] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:51:31 2022] GET /identity/v3/role_assignments?user.id=7679d070532c47fd97baee2e7f782dea&scope.project.id=b8d440e8301d40a78a6bb6029b17b2eb&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:33.876567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16444399-322b-4587-bfef-7b207d92848a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:33.877100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16444399-322b-4587-bfef-7b207d92848a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:33.877213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16444399-322b-4587-bfef-7b207d92848a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:33.878522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 415/840] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:33.884451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f98aa-c812-439e-b693-7432e7846aaf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:33.884865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f98aa-c812-439e-b693-7432e7846aaf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:33.884990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f98aa-c812-439e-b693-7432e7846aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:33.903583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-979f98aa-c812-439e-b693-7432e7846aaf None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:33.904794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-979f98aa-c812-439e-b693-7432e7846aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:33.968877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 426/841] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:34.254514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16444399-322b-4587-bfef-7b207d92848a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:34.317886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:34.351869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16444399-322b-4587-bfef-7b207d92848a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:34.385884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc95ff-30d3-427b-b638-c1048fbf73c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:34.386492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc95ff-30d3-427b-b638-c1048fbf73c2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:34.386765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc95ff-30d3-427b-b638-c1048fbf73c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:34.387028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc95ff-30d3-427b-b638-c1048fbf73c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:34.388174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc95ff-30d3-427b-b638-c1048fbf73c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:34.389584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc95ff-30d3-427b-b638-c1048fbf73c2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:34.391118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 416/842] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:34.397410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f98aa-c812-439e-b693-7432e7846aaf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:34.461172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:34.494002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f98aa-c812-439e-b693-7432e7846aaf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:34.527700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b66e7-040e-4ee3-9351-2fda50c17a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:34.528298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b66e7-040e-4ee3-9351-2fda50c17a90 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:34.528568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b66e7-040e-4ee3-9351-2fda50c17a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:34.533942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b66e7-040e-4ee3-9351-2fda50c17a90 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:34.533942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b66e7-040e-4ee3-9351-2fda50c17a90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:34.533942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b66e7-040e-4ee3-9351-2fda50c17a90 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:34.629085 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 427/843] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:51:34 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:36.799187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef448-e5bf-44b9-84d3-654d92c3d901 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:36.799722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef448-e5bf-44b9-84d3-654d92c3d901 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:36.799722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef448-e5bf-44b9-84d3-654d92c3d901 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:36.801066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 417/844] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:36.806372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b72f-1350-4961-9fc4-0dcea3191b04 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:36.806780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b72f-1350-4961-9fc4-0dcea3191b04 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:36.806909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b72f-1350-4961-9fc4-0dcea3191b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:36.825514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2019b72f-1350-4961-9fc4-0dcea3191b04 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:36.826701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2019b72f-1350-4961-9fc4-0dcea3191b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:37.004687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 428/845] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:37.270787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cef448-e5bf-44b9-84d3-654d92c3d901 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:37.371611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cef448-e5bf-44b9-84d3-654d92c3d901 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:37.405000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:37.405609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:37.405883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:37.406153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] PATH_INFO: `/v3/roles/a2f657dd4fab43288cb9e84eb6d94d51` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:37.408189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] RBAC: Authorizing `identity:get_role(role_id=a2f657dd4fab43288cb9e84eb6d94d51)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:37.409548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:37.410844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 418/846] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:37 2022] GET /identity/v3/roles/a2f657dd4fab43288cb9e84eb6d94d51 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:37.416719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019b72f-1350-4961-9fc4-0dcea3191b04 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:37.466785 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:37.511859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019b72f-1350-4961-9fc4-0dcea3191b04 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:37.544781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:37.545347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:37.545625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:37.545895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] PATH_INFO: `/v3/users/8e9fa5427538498f8b6c39ed7b6146cb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:37.559033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] RBAC: Authorizing `identity:get_user(user_id=8e9fa5427538498f8b6c39ed7b6146cb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:37.560482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:37.562034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 429/847] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:37 2022] GET /identity/v3/users/8e9fa5427538498f8b6c39ed7b6146cb => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:37.568329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:37.571129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:37.667881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2584ca8-d91a-4233-80f5-8d780dc7003d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:37.700849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87fbe7-d20e-4590-9136-22a6bef0c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:37.701468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87fbe7-d20e-4590-9136-22a6bef0c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:37.701747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87fbe7-d20e-4590-9136-22a6bef0c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:37.702020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87fbe7-d20e-4590-9136-22a6bef0c7c6 None admin] PATH_INFO: `/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:37.703125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87fbe7-d20e-4590-9136-22a6bef0c7c6 None admin] RBAC: Authorizing `identity:get_project(project_id=b8d440e8301d40a78a6bb6029b17b2eb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:37.705274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87fbe7-d20e-4590-9136-22a6bef0c7c6 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:37.706876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 419/848] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:51:37 2022] GET /identity/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:37.712734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:37.808636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e290e-1524-41cc-bb47-feea7ef9a87b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:37.819460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:37.843402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4062f-65a1-4187-9a52-a95d94871acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:37.844073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4062f-65a1-4187-9a52-a95d94871acd None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:37.844353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4062f-65a1-4187-9a52-a95d94871acd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:37.844723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4062f-65a1-4187-9a52-a95d94871acd None admin] PATH_INFO: `/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb/users/8e9fa5427538498f8b6c39ed7b6146cb/roles/a2f657dd4fab43288cb9e84eb6d94d51` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:37.847269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4062f-65a1-4187-9a52-a95d94871acd None admin] RBAC: Authorizing `identity:create_grant(project_id=b8d440e8301d40a78a6bb6029b17b2eb, user_id=8e9fa5427538498f8b6c39ed7b6146cb, role_id=a2f657dd4fab43288cb9e84eb6d94d51)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:37.848706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4062f-65a1-4187-9a52-a95d94871acd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:37.927424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 430/849] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:51:37 2022] PUT /identity/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb/users/8e9fa5427538498f8b6c39ed7b6146cb/roles/a2f657dd4fab43288cb9e84eb6d94d51 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:51:40.058417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c225d-048b-4da8-8256-d98a100ab780 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:40.058848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c225d-048b-4da8-8256-d98a100ab780 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:40.058970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c225d-048b-4da8-8256-d98a100ab780 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:40.060353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 420/850] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:40.065568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48034c-1ba4-49db-8d0d-df58a7ec2f26 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:40.065987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48034c-1ba4-49db-8d0d-df58a7ec2f26 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:40.066098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48034c-1ba4-49db-8d0d-df58a7ec2f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:40.083707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-aa48034c-1ba4-49db-8d0d-df58a7ec2f26 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:40.084904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aa48034c-1ba4-49db-8d0d-df58a7ec2f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:40.148704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 431/851] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:40.418652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c225d-048b-4da8-8256-d98a100ab780 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:40.486013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:40.515897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c225d-048b-4da8-8256-d98a100ab780 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:40.548697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:40.549291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:40.549565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:40.549835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] PATH_INFO: `/v3/roles/a2f657dd4fab43288cb9e84eb6d94d51` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:40.551186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] RBAC: Authorizing `identity:get_role(role_id=a2f657dd4fab43288cb9e84eb6d94d51)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:40.552575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:40.553857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 421/852] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:40 2022] GET /identity/v3/roles/a2f657dd4fab43288cb9e84eb6d94d51 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:40.559868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48034c-1ba4-49db-8d0d-df58a7ec2f26 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:40.653011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:40.655871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48034c-1ba4-49db-8d0d-df58a7ec2f26 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:40.689197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:40.689785 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:40.690061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:40.690329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] PATH_INFO: `/v3/users/8e9fa5427538498f8b6c39ed7b6146cb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:40.691430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9fa5427538498f8b6c39ed7b6146cb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:40.692870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:40.694427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 432/853] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:40 2022] GET /identity/v3/users/8e9fa5427538498f8b6c39ed7b6146cb => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:40.700267 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:40.795258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c0d4c-7188-41f7-9506-ad8de2eedcfb None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:40.827299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:40.827910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:40.828187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:40.828455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:40.836677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:40.893185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:40.894594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:40.987293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 422/854] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:51:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:51:40.993114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:41.089731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875558b-a895-49f1-bcaf-0a7e9a103a26 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:41.122459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e00ab-9115-4c0e-8917-cf2ca1324fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:41.123070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e00ab-9115-4c0e-8917-cf2ca1324fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:41.123343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e00ab-9115-4c0e-8917-cf2ca1324fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:41.123624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e00ab-9115-4c0e-8917-cf2ca1324fa2 None admin] PATH_INFO: `/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:41.124754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e00ab-9115-4c0e-8917-cf2ca1324fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=b8d440e8301d40a78a6bb6029b17b2eb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:41.126761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e00ab-9115-4c0e-8917-cf2ca1324fa2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:41.128386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 433/855] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:51:40 2022] GET /identity/v3/projects/b8d440e8301d40a78a6bb6029b17b2eb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:41.134882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:41.172392 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:41.229138 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edeceab-62a6-458a-be91-ffe99aecc2fe None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:41.261954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbed12-be5b-4c59-ad5e-75d67d1e1271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:41.262483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbed12-be5b-4c59-ad5e-75d67d1e1271 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:41.262755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbed12-be5b-4c59-ad5e-75d67d1e1271 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:41.263025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbed12-be5b-4c59-ad5e-75d67d1e1271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:41.263728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbed12-be5b-4c59-ad5e-75d67d1e1271 None admin] RBAC: Adding query filter params (role.id=a2f657dd4fab43288cb9e84eb6d94d51, scope.project.id=b8d440e8301d40a78a6bb6029b17b2eb, user.id=8e9fa5427538498f8b6c39ed7b6146cb) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:41.264060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbed12-be5b-4c59-ad5e-75d67d1e1271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:41.265466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbed12-be5b-4c59-ad5e-75d67d1e1271 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:41.271545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 423/856] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:51:41 2022] GET /identity/v3/role_assignments?user.id=8e9fa5427538498f8b6c39ed7b6146cb&scope.project.id=b8d440e8301d40a78a6bb6029b17b2eb&role.id=a2f657dd4fab43288cb9e84eb6d94d51 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:43.409022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16558c-e568-4e01-9ac6-fb7e89c348ac None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:43.409539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16558c-e568-4e01-9ac6-fb7e89c348ac None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:43.409539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16558c-e568-4e01-9ac6-fb7e89c348ac None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:43.410878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 434/857] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:43.416016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e66649-60f5-429d-9f63-d03ade94bbac None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:43.416389 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e66649-60f5-429d-9f63-d03ade94bbac None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:43.416562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e66649-60f5-429d-9f63-d03ade94bbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:43.434743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b6e66649-60f5-429d-9f63-d03ade94bbac None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:43.435958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b6e66649-60f5-429d-9f63-d03ade94bbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:43.500451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 424/858] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:43.770026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16558c-e568-4e01-9ac6-fb7e89c348ac None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:43.805704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:43.869633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16558c-e568-4e01-9ac6-fb7e89c348ac None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:43.901514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc812621-e1ca-401f-9b55-530f64e6d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:43.902125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc812621-e1ca-401f-9b55-530f64e6d656 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:43.902400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc812621-e1ca-401f-9b55-530f64e6d656 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:43.902667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc812621-e1ca-401f-9b55-530f64e6d656 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:43.903748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc812621-e1ca-401f-9b55-530f64e6d656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:43.905191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc812621-e1ca-401f-9b55-530f64e6d656 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:43.907252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 435/859] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:43.913160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e66649-60f5-429d-9f63-d03ade94bbac None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:43.942547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:44.008582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e66649-60f5-429d-9f63-d03ade94bbac None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:44.044583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d47ba-9ed3-42cd-814c-265ed0917f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:44.045189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d47ba-9ed3-42cd-814c-265ed0917f77 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:44.045472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d47ba-9ed3-42cd-814c-265ed0917f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:44.045741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d47ba-9ed3-42cd-814c-265ed0917f77 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:44.046390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d47ba-9ed3-42cd-814c-265ed0917f77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:44.048481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d47ba-9ed3-42cd-814c-265ed0917f77 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:44.134061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 425/860] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:43 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:51:46.515016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d8ac0-9943-4998-85be-457a08aef362 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:46.515546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d8ac0-9943-4998-85be-457a08aef362 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:46.515546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d8ac0-9943-4998-85be-457a08aef362 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:46.517250 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 436/861] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:46.522519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee9371-a3b1-4a46-a3e5-472b7023d7c1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:46.522923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee9371-a3b1-4a46-a3e5-472b7023d7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:46.523073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee9371-a3b1-4a46-a3e5-472b7023d7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:46.541030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a7ee9371-a3b1-4a46-a3e5-472b7023d7c1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:46.542252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a7ee9371-a3b1-4a46-a3e5-472b7023d7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:46.611754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 426/862] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:46.886121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d8ac0-9943-4998-85be-457a08aef362 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:46.985736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d8ac0-9943-4998-85be-457a08aef362 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:46.988175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:47.021710 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9c620-97b1-49d0-8a3e-bc55d03b6a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:47.022342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c620-97b1-49d0-8a3e-bc55d03b6a61 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:47.022621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c620-97b1-49d0-8a3e-bc55d03b6a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:47.022895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c620-97b1-49d0-8a3e-bc55d03b6a61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:47.024029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9c620-97b1-49d0-8a3e-bc55d03b6a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:47.025457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9c620-97b1-49d0-8a3e-bc55d03b6a61 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:47.027554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 437/863] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:51:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:47.033614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee9371-a3b1-4a46-a3e5-472b7023d7c1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:47.048812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:47.132553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee9371-a3b1-4a46-a3e5-472b7023d7c1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:47.167473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca8251-fff2-475e-b50f-ee3c06a143bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:47.168027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca8251-fff2-475e-b50f-ee3c06a143bf None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:47.168308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca8251-fff2-475e-b50f-ee3c06a143bf None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:47.168581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca8251-fff2-475e-b50f-ee3c06a143bf None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:47.169240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca8251-fff2-475e-b50f-ee3c06a143bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:47.170684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca8251-fff2-475e-b50f-ee3c06a143bf None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:47.268979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 427/864] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:51:47 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:49.480925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23afb-f6ac-4040-a043-c8b08f3a651f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:49.481454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23afb-f6ac-4040-a043-c8b08f3a651f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:49.481454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca23afb-f6ac-4040-a043-c8b08f3a651f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:49.482815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 438/865] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:49.489062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca28113-04fa-482a-85bb-7edc7cafe326 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:49.489489 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca28113-04fa-482a-85bb-7edc7cafe326 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:49.489618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca28113-04fa-482a-85bb-7edc7cafe326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:49.508057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-eca28113-04fa-482a-85bb-7edc7cafe326 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:49.509504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eca28113-04fa-482a-85bb-7edc7cafe326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:49.569551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:49.573808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 428/866] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:49.849999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23afb-f6ac-4040-a043-c8b08f3a651f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:49.947721 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca23afb-f6ac-4040-a043-c8b08f3a651f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:49.980388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:49.980995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:49.981269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:49.981535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:49.991826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:49.996493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:49.997577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:51:49.998745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 439/867] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:51:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:50.003928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca28113-04fa-482a-85bb-7edc7cafe326 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:50.108016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca28113-04fa-482a-85bb-7edc7cafe326 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:50.141363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:50.141950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:50.142224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:50.142490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:50.143135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:50.143432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:50.144821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:50.152505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 429/868] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:51:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:50.158263 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:50.189890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:50.255575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a02480-2f61-4ad9-8762-a2b6aa2dc0f5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:50.287252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:50.287885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:50.288162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:50.288430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] PATH_INFO: `/v3/users/f4717c9f292c45d19871b1fedfbf11bb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:50.301188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] RBAC: Authorizing `identity:get_user(user_id=f4717c9f292c45d19871b1fedfbf11bb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:50.302605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:50.304200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 440/869] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:50 2022] GET /identity/v3/users/f4717c9f292c45d19871b1fedfbf11bb => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:50.309933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:50.369185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:50.409880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33558449-6eb5-44b8-8b71-2e923d9bd65e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:50.442095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d45158-5427-4674-839d-914c7644cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:50.442709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45158-5427-4674-839d-914c7644cc91 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:50.442986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45158-5427-4674-839d-914c7644cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:50.443264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45158-5427-4674-839d-914c7644cc91 None admin] PATH_INFO: `/v3/projects/c8cbfd2c300542ddba9c96821eeab85d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:50.444371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d45158-5427-4674-839d-914c7644cc91 None admin] RBAC: Authorizing `identity:get_project(project_id=c8cbfd2c300542ddba9c96821eeab85d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:50.445794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d45158-5427-4674-839d-914c7644cc91 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:50.447383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 430/870] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:51:50 2022] GET /identity/v3/projects/c8cbfd2c300542ddba9c96821eeab85d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:50.453291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:50.538977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:50.548401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10875-e05b-4555-9af1-849494c0da7d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:50.580619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ffac2-af33-4416-8ab1-822f151b54b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:50.581251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ffac2-af33-4416-8ab1-822f151b54b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:50.581522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ffac2-af33-4416-8ab1-822f151b54b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:50.581790 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ffac2-af33-4416-8ab1-822f151b54b8 None admin] PATH_INFO: `/v3/projects/c8cbfd2c300542ddba9c96821eeab85d/users/f4717c9f292c45d19871b1fedfbf11bb/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:50.583621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ffac2-af33-4416-8ab1-822f151b54b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8cbfd2c300542ddba9c96821eeab85d, user_id=f4717c9f292c45d19871b1fedfbf11bb, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:50.585056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ffac2-af33-4416-8ab1-822f151b54b8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:50.645707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 441/871] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:51:50 2022] PUT /identity/v3/projects/c8cbfd2c300542ddba9c96821eeab85d/users/f4717c9f292c45d19871b1fedfbf11bb/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:51:52.766979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ac4af-723b-419e-8b83-cac9283b72ef None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:52.767508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ac4af-723b-419e-8b83-cac9283b72ef None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:52.767508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ac4af-723b-419e-8b83-cac9283b72ef None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:52.768892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 431/872] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:52.774255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf41b7-bc8e-4729-868f-635d1b105080 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:52.774686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf41b7-bc8e-4729-868f-635d1b105080 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:52.774786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf41b7-bc8e-4729-868f-635d1b105080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:52.793340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e2bf41b7-bc8e-4729-868f-635d1b105080 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:52.794551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e2bf41b7-bc8e-4729-868f-635d1b105080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:52.859860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 442/873] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:53.131675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ac4af-723b-419e-8b83-cac9283b72ef None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:53.229037 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ac4af-723b-419e-8b83-cac9283b72ef None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:53.256856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:53.262618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:53.263230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:53.263505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:53.263772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:53.273475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:51:53.278312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:51:53.279457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:51:53.280560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 432/874] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:51:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:51:53.285500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf41b7-bc8e-4729-868f-635d1b105080 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:53.363194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:53.381468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf41b7-bc8e-4729-868f-635d1b105080 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:53.414474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:53.415128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:53.415408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:53.415680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:53.416361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:53.416663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:53.418012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:53.426753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 443/875] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:51:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:51:53.432380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:53.531698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91581f6d-6339-4d93-8f5c-f1f7ba56b590 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:53.565035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:53.565579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:53.565857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:53.566126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] PATH_INFO: `/v3/users/f4717c9f292c45d19871b1fedfbf11bb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:53.567235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] RBAC: Authorizing `identity:get_user(user_id=f4717c9f292c45d19871b1fedfbf11bb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:53.568696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:53.570264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 433/876] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:51:53 2022] GET /identity/v3/users/f4717c9f292c45d19871b1fedfbf11bb => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:53.577373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:53.683551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33118b62-0676-4f78-9c0f-16944e049c58 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:53.717666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:53.719179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:53.719479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:53.719746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:53.735059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:53.794569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:53.796036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:53.899577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 444/877] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:51:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:51:53.905793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:53.965641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:54.010174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf62d1-ddc5-48ad-a70e-cfedb7a28f35 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:54.055628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8148b-a079-4b31-aea9-d68bd9935306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:54.056274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8148b-a079-4b31-aea9-d68bd9935306 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:54.056549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8148b-a079-4b31-aea9-d68bd9935306 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:54.056817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8148b-a079-4b31-aea9-d68bd9935306 None admin] PATH_INFO: `/v3/projects/c8cbfd2c300542ddba9c96821eeab85d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:54.057911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8148b-a079-4b31-aea9-d68bd9935306 None admin] RBAC: Authorizing `identity:get_project(project_id=c8cbfd2c300542ddba9c96821eeab85d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:54.059337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8148b-a079-4b31-aea9-d68bd9935306 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:54.060990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 434/878] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:51:53 2022] GET /identity/v3/projects/c8cbfd2c300542ddba9c96821eeab85d => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:54.067901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:54.114768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:54.174661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaf810-e59e-4bf7-8e05-3e9bf0ccabcd None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:54.209255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403735a8-87b9-4db0-aa81-44a98d164543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:54.209853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403735a8-87b9-4db0-aa81-44a98d164543 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:54.210128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403735a8-87b9-4db0-aa81-44a98d164543 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:54.210396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403735a8-87b9-4db0-aa81-44a98d164543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:54.211100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403735a8-87b9-4db0-aa81-44a98d164543 None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=c8cbfd2c300542ddba9c96821eeab85d, user.id=f4717c9f292c45d19871b1fedfbf11bb) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:51:54.211407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403735a8-87b9-4db0-aa81-44a98d164543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:54.212845 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403735a8-87b9-4db0-aa81-44a98d164543 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:54.219609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 445/879] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:51:54 2022] GET /identity/v3/role_assignments?user.id=f4717c9f292c45d19871b1fedfbf11bb&scope.project.id=c8cbfd2c300542ddba9c96821eeab85d&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:56.371196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50fe8-8281-4084-84f0-e9aef54c3130 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:56.371730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50fe8-8281-4084-84f0-e9aef54c3130 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:56.371730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50fe8-8281-4084-84f0-e9aef54c3130 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:56.373131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 435/880] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:56.379289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47be16-6ebd-4a63-be37-8257c2ad3f56 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:56.379688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47be16-6ebd-4a63-be37-8257c2ad3f56 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:56.379875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47be16-6ebd-4a63-be37-8257c2ad3f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:56.398405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5e47be16-6ebd-4a63-be37-8257c2ad3f56 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:56.399577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5e47be16-6ebd-4a63-be37-8257c2ad3f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:56.464475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 446/881] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:56.731844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50fe8-8281-4084-84f0-e9aef54c3130 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:56.831465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50fe8-8281-4084-84f0-e9aef54c3130 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:56.855979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:56.864673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6962b8-8b9c-4f52-ba2f-ece864893cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:56.871314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6962b8-8b9c-4f52-ba2f-ece864893cb4 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:56.871314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6962b8-8b9c-4f52-ba2f-ece864893cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:56.871314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6962b8-8b9c-4f52-ba2f-ece864893cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:56.871314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6962b8-8b9c-4f52-ba2f-ece864893cb4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:56.871314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6962b8-8b9c-4f52-ba2f-ece864893cb4 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:56.949973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 436/882] 10.222.0.20 () {60 vars in 1138 bytes} [Wed Oct 26 10:51:56 2022] POST /identity/v3/domains => generated 267 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:51:59.121347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcef25c-0a26-42f7-8df4-9e76e081484b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:59.121889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcef25c-0a26-42f7-8df4-9e76e081484b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:59.121889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcef25c-0a26-42f7-8df4-9e76e081484b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:59.123232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 447/883] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:51:59.129050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b93a8-76ff-4a7b-b691-28df40330b29 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:59.129483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b93a8-76ff-4a7b-b691-28df40330b29 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:59.129600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b93a8-76ff-4a7b-b691-28df40330b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:59.147993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8d9b93a8-76ff-4a7b-b691-28df40330b29 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:51:59.149183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8d9b93a8-76ff-4a7b-b691-28df40330b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:51:59.214246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 437/884] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:51:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:51:59.485165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcef25c-0a26-42f7-8df4-9e76e081484b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:59.519970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:59.582119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcef25c-0a26-42f7-8df4-9e76e081484b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:59.614858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc04f65-0957-48c6-97f6-5394f6aa43ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:59.615474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04f65-0957-48c6-97f6-5394f6aa43ef None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:59.615748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04f65-0957-48c6-97f6-5394f6aa43ef None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:59.616045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04f65-0957-48c6-97f6-5394f6aa43ef None admin] PATH_INFO: `/v3/domains/7c9e9450da8e4308b32fd33b0a2476ae` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:59.625927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc04f65-0957-48c6-97f6-5394f6aa43ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c9e9450da8e4308b32fd33b0a2476ae)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:59.627341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc04f65-0957-48c6-97f6-5394f6aa43ef None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:59.628914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 448/885] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Oct 26 10:51:59 2022] GET /identity/v3/domains/7c9e9450da8e4308b32fd33b0a2476ae => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:51:59.634927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b93a8-76ff-4a7b-b691-28df40330b29 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:51:59.729866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b93a8-76ff-4a7b-b691-28df40330b29 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:51:59.746332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:51:59.763541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9674a67-ba72-485c-b638-ba8a2c894096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:51:59.764171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674a67-ba72-485c-b638-ba8a2c894096 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:51:59.764451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674a67-ba72-485c-b638-ba8a2c894096 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:51:59.764726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674a67-ba72-485c-b638-ba8a2c894096 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:51:59.765929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9674a67-ba72-485c-b638-ba8a2c894096 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:51:59.767345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9674a67-ba72-485c-b638-ba8a2c894096 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:51:59.846204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 438/886] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:51:59 2022] POST /identity/v3/projects => generated 361 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:52:02.011807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20f72f-2ab2-4756-b10b-5b626faf44c6 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:02.012447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20f72f-2ab2-4756-b10b-5b626faf44c6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:02.012447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20f72f-2ab2-4756-b10b-5b626faf44c6 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:02.013709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 449/887] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:52:02.019708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e319f-3c98-4583-a5de-82fa3edab3af None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:02.020234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e319f-3c98-4583-a5de-82fa3edab3af None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:02.020327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e319f-3c98-4583-a5de-82fa3edab3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:02.038425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7b2e319f-3c98-4583-a5de-82fa3edab3af None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:52:02.039615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7b2e319f-3c98-4583-a5de-82fa3edab3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:52:02.106533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 439/888] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:52:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:52:02.389859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20f72f-2ab2-4756-b10b-5b626faf44c6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:02.481959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:02.489031 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20f72f-2ab2-4756-b10b-5b626faf44c6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:02.522254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953fd13-321e-4a60-ab1b-878d357b627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:02.522872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953fd13-321e-4a60-ab1b-878d357b627b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:02.523152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953fd13-321e-4a60-ab1b-878d357b627b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:02.523424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953fd13-321e-4a60-ab1b-878d357b627b None admin] PATH_INFO: `/v3/domains/7c9e9450da8e4308b32fd33b0a2476ae` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:02.524551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953fd13-321e-4a60-ab1b-878d357b627b None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c9e9450da8e4308b32fd33b0a2476ae)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:02.525969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953fd13-321e-4a60-ab1b-878d357b627b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:02.527507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 450/889] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Oct 26 10:52:02 2022] GET /identity/v3/domains/7c9e9450da8e4308b32fd33b0a2476ae => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:02.533840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e319f-3c98-4583-a5de-82fa3edab3af None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:02.625988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:02.629394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e319f-3c98-4583-a5de-82fa3edab3af None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:02.662919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd61267-3e6c-4836-9e0a-7a1447194431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:02.663455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd61267-3e6c-4836-9e0a-7a1447194431 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:02.663731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd61267-3e6c-4836-9e0a-7a1447194431 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:02.664103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd61267-3e6c-4836-9e0a-7a1447194431 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:02.672239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd61267-3e6c-4836-9e0a-7a1447194431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:02.672239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd61267-3e6c-4836-9e0a-7a1447194431 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:02.742604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 440/890] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:52:02 2022] POST /identity/v3/users => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:52:04.900533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be6492-d4a9-41cd-b6df-4990d004526c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:04.901095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be6492-d4a9-41cd-b6df-4990d004526c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:04.901095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be6492-d4a9-41cd-b6df-4990d004526c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:04.902385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 451/891] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:52:04.908042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec04b5-9372-4b67-b9f0-a07d582ee5a2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:04.908437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec04b5-9372-4b67-b9f0-a07d582ee5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:04.908602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec04b5-9372-4b67-b9f0-a07d582ee5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:04.927067 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-55ec04b5-9372-4b67-b9f0-a07d582ee5a2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:52:04.928562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-55ec04b5-9372-4b67-b9f0-a07d582ee5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:52:04.992399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 441/892] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:52:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:52:05.263499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be6492-d4a9-41cd-b6df-4990d004526c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:05.363558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be6492-d4a9-41cd-b6df-4990d004526c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:05.396393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:05.397012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:05.397290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:05.397560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:05.407922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:52:05.412681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:52:05.413774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:52:05.414752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 452/893] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:52:05.419831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec04b5-9372-4b67-b9f0-a07d582ee5a2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:05.500964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:05.515961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec04b5-9372-4b67-b9f0-a07d582ee5a2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:05.548841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:05.549439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:05.549719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:05.549990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:05.550634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:52:05.550933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:05.552335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:05.560319 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 442/894] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:05.566289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:05.591494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:05.662805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a95cd2-7ced-420c-b794-59e0ff659099 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:05.695459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:05.696097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:05.696372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:05.696641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] PATH_INFO: `/v3/users/f4e6280daf3a4c51933b5234f43b913f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:05.710468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] RBAC: Authorizing `identity:get_user(user_id=f4e6280daf3a4c51933b5234f43b913f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:05.711903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:05.713459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 453/895] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:52:05 2022] GET /identity/v3/users/f4e6280daf3a4c51933b5234f43b913f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:05.719231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:05.818154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388954-60ff-4212-b400-b36b885703a2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:05.850451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f365c00-10f4-4e19-a7b7-b0f4ae161b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:05.851066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365c00-10f4-4e19-a7b7-b0f4ae161b0a None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:05.851346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365c00-10f4-4e19-a7b7-b0f4ae161b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:05.851620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365c00-10f4-4e19-a7b7-b0f4ae161b0a None admin] PATH_INFO: `/v3/projects/0bf94d89ed924436bf0adfa9d546e8f4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:05.852767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f365c00-10f4-4e19-a7b7-b0f4ae161b0a None admin] RBAC: Authorizing `identity:get_project(project_id=0bf94d89ed924436bf0adfa9d546e8f4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:05.854226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f365c00-10f4-4e19-a7b7-b0f4ae161b0a None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:05.855844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 443/896] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:52:05 2022] GET /identity/v3/projects/0bf94d89ed924436bf0adfa9d546e8f4 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:05.862193 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:05.940085 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:05.956979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8add7f-8333-4418-8255-8a058b16e922 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:05.989573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db7c30-ec47-450e-ae34-799a0c61d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:05.990212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7c30-ec47-450e-ae34-799a0c61d929 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:05.990489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7c30-ec47-450e-ae34-799a0c61d929 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:05.990759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7c30-ec47-450e-ae34-799a0c61d929 None admin] PATH_INFO: `/v3/projects/0bf94d89ed924436bf0adfa9d546e8f4/users/f4e6280daf3a4c51933b5234f43b913f/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:05.992781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db7c30-ec47-450e-ae34-799a0c61d929 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bf94d89ed924436bf0adfa9d546e8f4, user_id=f4e6280daf3a4c51933b5234f43b913f, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:05.994182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db7c30-ec47-450e-ae34-799a0c61d929 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:06.057178 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 454/897] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:52:05 2022] PUT /identity/v3/projects/0bf94d89ed924436bf0adfa9d546e8f4/users/f4e6280daf3a4c51933b5234f43b913f/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:52:08.172584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff985-d2d6-4899-81d7-c89b9424d8ca None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:08.173139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff985-d2d6-4899-81d7-c89b9424d8ca None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:08.173139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff985-d2d6-4899-81d7-c89b9424d8ca None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:08.174445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 444/898] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:52:08.179589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b559a6-bef9-4ac1-9ca4-43129462413a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:08.180039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b559a6-bef9-4ac1-9ca4-43129462413a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:08.180183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b559a6-bef9-4ac1-9ca4-43129462413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:08.198794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a2b559a6-bef9-4ac1-9ca4-43129462413a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:52:08.200003 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a2b559a6-bef9-4ac1-9ca4-43129462413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:52:08.264925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 455/899] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:52:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:52:08.539167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff985-d2d6-4899-81d7-c89b9424d8ca None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:08.555240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:08.641224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff985-d2d6-4899-81d7-c89b9424d8ca None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:08.673548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:08.674157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:08.674432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:08.674700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:08.684240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:52:08.689509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:52:08.690594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:52:08.691609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 445/900] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:52:08.696787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b559a6-bef9-4ac1-9ca4-43129462413a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:08.764097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:08.792299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b559a6-bef9-4ac1-9ca4-43129462413a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:08.825613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:08.826207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:08.826484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:08.826753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:08.827635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:52:08.827965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:08.829318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:08.837587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 456/901] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:08.843460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:08.916255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:08.940730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5d53-65ee-48e6-91f4-9e064b78f5cd None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:08.973555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:08.974162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:08.974440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:08.974712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] PATH_INFO: `/v3/users/f4e6280daf3a4c51933b5234f43b913f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:08.975873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] RBAC: Authorizing `identity:get_user(user_id=f4e6280daf3a4c51933b5234f43b913f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:08.977301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:08.978844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 446/902] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 26 10:52:08 2022] GET /identity/v3/users/f4e6280daf3a4c51933b5234f43b913f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:08.985137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:09.081776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4afb1-908c-4e27-a2e6-70f607eba1fe None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:09.114520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:09.115108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:09.115383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:09.115650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:09.118136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:09.181501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:09.182935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:09.276384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 457/903] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:52:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:52:09.282829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:09.378463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2a979-c156-4d0b-a70b-b1ea0c898550 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:09.411702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e9d08-1b39-4c4f-a306-9c3615c52d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:09.412334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e9d08-1b39-4c4f-a306-9c3615c52d43 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:09.412608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e9d08-1b39-4c4f-a306-9c3615c52d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:09.412877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e9d08-1b39-4c4f-a306-9c3615c52d43 None admin] PATH_INFO: `/v3/projects/0bf94d89ed924436bf0adfa9d546e8f4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:09.413966 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e9d08-1b39-4c4f-a306-9c3615c52d43 None admin] RBAC: Authorizing `identity:get_project(project_id=0bf94d89ed924436bf0adfa9d546e8f4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:09.415383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e9d08-1b39-4c4f-a306-9c3615c52d43 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:09.417008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 447/904] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 26 10:52:09 2022] GET /identity/v3/projects/0bf94d89ed924436bf0adfa9d546e8f4 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:09.423264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:52:09.456640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:09.520285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e6244-1a12-4b8b-9b67-633ec49363ac None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:52:09.552752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04099d0-8bce-40c4-a308-4e7985b06d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:52:09.553333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04099d0-8bce-40c4-a308-4e7985b06d1e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:52:09.553608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04099d0-8bce-40c4-a308-4e7985b06d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:52:09.553878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04099d0-8bce-40c4-a308-4e7985b06d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:52:09.554580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04099d0-8bce-40c4-a308-4e7985b06d1e None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=0bf94d89ed924436bf0adfa9d546e8f4, user.id=f4e6280daf3a4c51933b5234f43b913f) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:52:09.554881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04099d0-8bce-40c4-a308-4e7985b06d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:52:09.556315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04099d0-8bce-40c4-a308-4e7985b06d1e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:52:09.561869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 458/905] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:52:09 2022] GET /identity/v3/role_assignments?user.id=f4e6280daf3a4c51933b5234f43b913f&scope.project.id=0bf94d89ed924436bf0adfa9d546e8f4&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:52:19.407723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:52:19.559285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=51 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:27.907133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a00ba-0867-4c14-983b-17bf73876774 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:27.907719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a00ba-0867-4c14-983b-17bf73876774 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:27.907719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a00ba-0867-4c14-983b-17bf73876774 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:27.909053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 448/906] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:27.914892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86be32-adfb-40c7-b187-56c86c24d5aa None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:27.915357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86be32-adfb-40c7-b187-56c86c24d5aa None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:27.915464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86be32-adfb-40c7-b187-56c86c24d5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:27.933592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6d86be32-adfb-40c7-b187-56c86c24d5aa None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:27.934739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6d86be32-adfb-40c7-b187-56c86c24d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:27.998811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 459/907] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:28.274204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a00ba-0867-4c14-983b-17bf73876774 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:28.375150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a00ba-0867-4c14-983b-17bf73876774 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:28.408847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:28.409463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:28.409740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:28.410046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:28.419731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:28.421199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:28.436668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:28.440285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:28.441468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:28.441468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:28.441468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:28.441468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 449/908] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:28.444086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86be32-adfb-40c7-b187-56c86c24d5aa None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:28.540762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86be32-adfb-40c7-b187-56c86c24d5aa None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:28.568167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:28.574574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd378a-5d71-449f-83d8-01874d7b661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:28.575174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd378a-5d71-449f-83d8-01874d7b661a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:28.575451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd378a-5d71-449f-83d8-01874d7b661a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:28.575724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd378a-5d71-449f-83d8-01874d7b661a None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:28.576384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd378a-5d71-449f-83d8-01874d7b661a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:28.576683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd378a-5d71-449f-83d8-01874d7b661a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:28.578034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd378a-5d71-449f-83d8-01874d7b661a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:28.589434 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 460/909] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:28.595237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:28.648265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:28.694218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bd9a5-5e8b-4a30-8f1e-64e47be5cdd0 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:28.729403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48268c7c-f681-4e48-9c8c-ff3e868f0a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:28.730033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268c7c-f681-4e48-9c8c-ff3e868f0a52 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:28.730309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268c7c-f681-4e48-9c8c-ff3e868f0a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:28.730580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268c7c-f681-4e48-9c8c-ff3e868f0a52 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:28.731232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48268c7c-f681-4e48-9c8c-ff3e868f0a52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:28.732682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48268c7c-f681-4e48-9c8c-ff3e868f0a52 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:28.822627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 450/910] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 26 10:53:28 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:53:31.014286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64984c8d-abc5-46aa-99a0-08fcd8ae08d4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:31.014846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64984c8d-abc5-46aa-99a0-08fcd8ae08d4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:31.014846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64984c8d-abc5-46aa-99a0-08fcd8ae08d4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:31.016355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 461/911] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:31.021631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68728f5b-297c-4f6c-aaf4-000f00fc54f7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:31.022041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68728f5b-297c-4f6c-aaf4-000f00fc54f7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:31.022186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68728f5b-297c-4f6c-aaf4-000f00fc54f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:31.041756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-68728f5b-297c-4f6c-aaf4-000f00fc54f7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:31.043613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-68728f5b-297c-4f6c-aaf4-000f00fc54f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:31.111728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 451/912] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:31.384345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64984c8d-abc5-46aa-99a0-08fcd8ae08d4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:31.485041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64984c8d-abc5-46aa-99a0-08fcd8ae08d4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:31.517274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:31.517891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:31.518165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:31.518433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:31.529214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:53:31.534019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:31.535120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:31.542012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 462/913] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:53:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:31.542900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68728f5b-297c-4f6c-aaf4-000f00fc54f7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:31.581160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:31.638972 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68728f5b-297c-4f6c-aaf4-000f00fc54f7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:31.672418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:31.673007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:31.673284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:31.677660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:31.677660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] RBAC: Adding query filter params (name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:31.677660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:31.677660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:31.683688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 452/914] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:53:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:31.689400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:31.739338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:31.789574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c9550-71f0-4ce8-a43f-d2d0f2cd9e97 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:31.822097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:31.822717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:31.822998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:31.823273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:31.834506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:31.837117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:31.850992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:31.856140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:31.857273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:31.857273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:31.857273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:31.857273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 463/915] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:31.857857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:31.955872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:31.958728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f29d46-5a1d-467e-9d90-704e7055f609 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:31.992094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:31.992716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:31.992995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:31.993309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:31.993940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:31.994260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:31.995625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:32.007111 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 453/916] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:32.013038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:32.110956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113c9a-4314-48ec-b451-5394b4334d7a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:32.112505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:32.145040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:32.148137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:32.148137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:32.148137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:32.153416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:32.154826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:53:32.165876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:32.166977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:32.167938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 464/917] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:32.172840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:32.271146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d77450-ab11-4d05-9ffb-b8e338af3b5e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:32.304268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:32.305052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:32.305321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:32.305587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:32.306293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:32.306588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:32.308033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:32.326882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 454/918] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 26 10:53:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:32.332774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:32.430466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9c8ce-4df6-4393-b1b6-a74f64bff3ff None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:32.464502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:32.465124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:32.465402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:32.465674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:32.476425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:32.477855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:32.492078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:32.493255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:32.494646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:32.494646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:32.494646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:32.494646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 465/919] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:32.498992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:32.518448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:32.597911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913ff46-e7d3-4c10-8800-798d42b58b48 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:32.631942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:32.632548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:32.632829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:32.633097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:32.633727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:32.634028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:32.635387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:32.646189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 455/920] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:32.652282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:32.668714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:32.749896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d263cc-2a6b-4dc8-abb1-e396f9784e25 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:32.782989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:32.783606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:32.783917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:32.784192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:32.791086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:32.792572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:53:32.804066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:32.805185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:32.806472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 466/921] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:53:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:32.811190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:32.887140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:32.908367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f759efc-9675-4864-a200-457c15a668f9 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:32.942203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286cc4c-9a4c-41ad-96a9-4a9e7b889c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:32.942800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cc4c-9a4c-41ad-96a9-4a9e7b889c95 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:32.943078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cc4c-9a4c-41ad-96a9-4a9e7b889c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:32.943350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cc4c-9a4c-41ad-96a9-4a9e7b889c95 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:32.944088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286cc4c-9a4c-41ad-96a9-4a9e7b889c95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:32.944418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286cc4c-9a4c-41ad-96a9-4a9e7b889c95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:32.945825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286cc4c-9a4c-41ad-96a9-4a9e7b889c95 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:32.957473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 456/922] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:53:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:32.964006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:33.038698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:33.063090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba53784-a822-4aea-b870-99d347c3d8ba None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:33.095500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8761e02-104a-41f5-a329-b619b91696a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:33.096170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8761e02-104a-41f5-a329-b619b91696a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:33.096451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8761e02-104a-41f5-a329-b619b91696a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:33.096717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8761e02-104a-41f5-a329-b619b91696a7 None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/4c154983aff14d9895fb4b542db61e79/roles/7f166188476b48158de248c89d0b9144` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:33.111191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8761e02-104a-41f5-a329-b619b91696a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=4c154983aff14d9895fb4b542db61e79, role_id=7f166188476b48158de248c89d0b9144)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:33.112632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8761e02-104a-41f5-a329-b619b91696a7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:33.182230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 467/923] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:53:32 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/4c154983aff14d9895fb4b542db61e79/roles/7f166188476b48158de248c89d0b9144 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:53:35.378265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135953-e6b4-4e81-bc1e-aa036d184bc1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:35.378791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135953-e6b4-4e81-bc1e-aa036d184bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:35.378791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135953-e6b4-4e81-bc1e-aa036d184bc1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:35.380540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 457/924] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:35.386679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb35ac-128d-457e-a3d7-93a3dac9827b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:35.387077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb35ac-128d-457e-a3d7-93a3dac9827b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:35.387208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb35ac-128d-457e-a3d7-93a3dac9827b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:35.405792 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7bcb35ac-128d-457e-a3d7-93a3dac9827b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:35.407005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7bcb35ac-128d-457e-a3d7-93a3dac9827b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:35.472825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 468/925] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:35.747435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51135953-e6b4-4e81-bc1e-aa036d184bc1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:35.848173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51135953-e6b4-4e81-bc1e-aa036d184bc1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:35.882401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:35.883054 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:35.883339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:35.883647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:35.890275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:35.893929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:53:35.899503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:35.900730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:35.901873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 458/926] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 26 10:53:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:35.906749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb35ac-128d-457e-a3d7-93a3dac9827b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:35.959294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:36.003407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb35ac-128d-457e-a3d7-93a3dac9827b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:36.036868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:36.037502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:36.038495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:36.038495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:36.039261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:36.039559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:36.040940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:36.049034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 469/927] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 26 10:53:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:36.054895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:36.154323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fe2c2-b10e-4e94-88ad-c42d366ee15e None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:36.187781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:36.188465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:36.188746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:36.189016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:36.198853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:36.200314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:36.215021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:36.220122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:36.221179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:36.221179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:36.221179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:36.221179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 459/928] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:36.222075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:36.321780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636e7cb-8d18-4cb3-8e88-f8836f0d444c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:36.337625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:36.354378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:36.354968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:36.355244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:36.355510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:36.356168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:36.356464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:36.357830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:36.369000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 470/929] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:36.374641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:36.425143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:36.473129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf232b-8ca6-4d8a-91f6-a9f7bb76d31d None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:36.504715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:36.505325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:36.505600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:36.505864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:36.513759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:36.515196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:53:36.526492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:36.527561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:36.528877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 460/930] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:36.533348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:36.628779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd3c2e-301f-4b61-8fd7-19cdaa4ccc34 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:36.662438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:36.668512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:36.668512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:36.668512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:36.668512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:36.668512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:36.668512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:36.683888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 471/931] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 26 10:53:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:36.689637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:36.786302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e694ba-2090-45f7-b38d-ac34d1494d3c None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:36.797147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:36.820421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:36.821023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:36.821283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:36.821549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:36.886916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:36.888320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:36.983415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 461/932] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:53:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:53:36.989350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:37.024851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:37.089771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7ed11-5c28-4733-8e2a-5c07ae304a2b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:37.123210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:37.123881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:37.124163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:37.124436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:37.134614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:37.136115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:37.152282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:37.153427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:37.154525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:37.154525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:37.154525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:37.154525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 472/933] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:37.159219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:37.171256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:37.257810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1f09-4763-4326-87e0-4a1abf00f804 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:37.291544 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:37.292170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:37.292448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:37.292716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:37.293410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:37.293710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:37.295187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:37.306866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 462/934] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:37.312566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:37.399235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:37.414302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b1482-3708-48b0-b1c8-1ca186cbb33d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:37.445963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:37.446573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:37.446873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:37.447139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:37.453538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:37.454952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:53:37.466643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:37.471616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:37.472967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 473/935] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:37.473645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:37.544172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:37.574523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9a306-b311-4522-9baf-0604fd718916 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:37.608216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e435ee-bf56-4a56-b40c-f96e2e46f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:37.608808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e435ee-bf56-4a56-b40c-f96e2e46f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:37.609083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e435ee-bf56-4a56-b40c-f96e2e46f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:37.609356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e435ee-bf56-4a56-b40c-f96e2e46f4e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:37.610029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e435ee-bf56-4a56-b40c-f96e2e46f4e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:37.610333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e435ee-bf56-4a56-b40c-f96e2e46f4e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:37.611753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e435ee-bf56-4a56-b40c-f96e2e46f4e2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:37.624554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 463/936] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:53:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:53:37.630975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:37.730083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190499-cb4b-494e-baec-287c9ab5eafa None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:37.763609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dff7a1-fc8d-4f86-a328-a9094041342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:37.764223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dff7a1-fc8d-4f86-a328-a9094041342f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:37.764497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dff7a1-fc8d-4f86-a328-a9094041342f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:37.764762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dff7a1-fc8d-4f86-a328-a9094041342f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:37.765461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dff7a1-fc8d-4f86-a328-a9094041342f None admin] RBAC: Adding query filter params (role.id=7f166188476b48158de248c89d0b9144, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=4c154983aff14d9895fb4b542db61e79) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:37.765791 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dff7a1-fc8d-4f86-a328-a9094041342f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:37.767187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dff7a1-fc8d-4f86-a328-a9094041342f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:37.773996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 474/937] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:53:37 2022] GET /identity/v3/role_assignments?user.id=4c154983aff14d9895fb4b542db61e79&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=7f166188476b48158de248c89d0b9144 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:39.949339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4441c7d-d9a3-439e-8b26-05e87a4cb84f None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:39.949871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4441c7d-d9a3-439e-8b26-05e87a4cb84f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:39.949871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4441c7d-d9a3-439e-8b26-05e87a4cb84f None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:39.951217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 464/938] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:39.956279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913641cd-77ab-4f33-976b-e27525562716 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:39.956758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913641cd-77ab-4f33-976b-e27525562716 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:39.956879 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913641cd-77ab-4f33-976b-e27525562716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:39.975435 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-913641cd-77ab-4f33-976b-e27525562716 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:39.976705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-913641cd-77ab-4f33-976b-e27525562716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:39.979358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:40.043533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 475/939] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:40.325390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4441c7d-d9a3-439e-8b26-05e87a4cb84f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:40.426691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4441c7d-d9a3-439e-8b26-05e87a4cb84f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:40.455742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:40.461866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:40.462482 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:40.462762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:40.463032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:40.472723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:53:40.477618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:40.478696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:40.479587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 465/940] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:53:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:40.484815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913641cd-77ab-4f33-976b-e27525562716 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:40.581995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913641cd-77ab-4f33-976b-e27525562716 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:40.614345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:40.615004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:40.615279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:40.616343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:40.616737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:40.617032 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:40.618370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:40.626180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 476/941] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:53:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:40.632925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:40.731592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbebf72-8b16-46da-b350-78bbb81b3db1 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:40.765247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:40.765968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:40.766251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:40.766520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:40.776299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:40.777729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:40.792802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:40.795294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:40.796369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:40.796369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:40.796369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:40.796369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 466/942] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:40.799941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:40.802921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:40.898364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f73ac-1bf4-4499-92e0-34b1e49f9a73 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:40.931626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:40.932282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:40.932558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:40.932823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:40.933444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:40.933761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:40.935162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:40.947465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 477/943] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:40.953186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:40.996937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:41.052147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb259e-7657-419d-96e7-2bf1e68b2588 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:41.085267 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:41.085892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:41.086170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:41.086443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:41.093720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:41.095193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:53:41.107131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:41.108244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:41.109343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 467/944] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:41.114114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:41.175578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:41.212701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdb04b-5200-4909-9d9e-cb64b909e786 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:41.246069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:41.246659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:41.246932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:41.247198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:41.247958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:41.248255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:41.249717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:41.266478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 478/945] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 26 10:53:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:41.272996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:41.371614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bdc6c-67b9-4676-bad5-314cdeecb4f7 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:41.373125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:41.406270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:41.406885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:41.407161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:41.407434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:41.417814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:41.419258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:41.440999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:41.441990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:41.442901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:41.442901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:41.442901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:41.442901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 468/946] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:41.443200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:41.537987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106af6e-e9ca-4220-a151-db6ec7d45cdb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:41.557202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:41.571007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:41.571600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:41.571905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:41.572177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:41.572803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:41.573101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:41.574445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:41.587071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 479/947] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:41.592756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:41.692075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18732ffd-25a6-4e7f-9e1b-95c6c33c3f5b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:41.726584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:41.727209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:41.727483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:41.727749 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:41.734404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:41.735861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:53:41.747956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:41.748941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:41.749895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 469/948] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:53:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:41.755302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:41.854673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120d435-8d39-41d0-aaa3-04fa488c3a19 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:41.888936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2028a-c073-4bf0-9bd4-4025ae911c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:41.889538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2028a-c073-4bf0-9bd4-4025ae911c7b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:41.889817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2028a-c073-4bf0-9bd4-4025ae911c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:41.890091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2028a-c073-4bf0-9bd4-4025ae911c7b None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:41.890770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2028a-c073-4bf0-9bd4-4025ae911c7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:41.891074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2028a-c073-4bf0-9bd4-4025ae911c7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:41.892526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2028a-c073-4bf0-9bd4-4025ae911c7b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:41.904431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 480/949] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:53:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:53:41.910540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:41.926832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:42.009761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ceaa1-09db-4753-80de-9546245a8a0b None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:42.042849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52f5fb-47e7-4408-9664-ed461bfcaa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:42.043549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f5fb-47e7-4408-9664-ed461bfcaa44 None admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:42.043856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f5fb-47e7-4408-9664-ed461bfcaa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:42.044133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f5fb-47e7-4408-9664-ed461bfcaa44 None admin] PATH_INFO: `/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/4c154983aff14d9895fb4b542db61e79/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:42.046007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52f5fb-47e7-4408-9664-ed461bfcaa44 None admin] RBAC: Authorizing `identity:create_grant(project_id=5968f35bae3d4643bc199a16ab96770b, user_id=4c154983aff14d9895fb4b542db61e79, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:42.047422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52f5fb-47e7-4408-9664-ed461bfcaa44 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:42.204184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 470/950] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 26 10:53:41 2022] PUT /identity/v3/projects/5968f35bae3d4643bc199a16ab96770b/users/4c154983aff14d9895fb4b542db61e79/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:53:44.346806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f575-5cc6-40b5-88e8-641d85bd4fc0 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:44.347342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f575-5cc6-40b5-88e8-641d85bd4fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:44.347342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f575-5cc6-40b5-88e8-641d85bd4fc0 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:44.348637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 481/951] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:44.354071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227afaf-0692-4bb4-a733-b6a350ea2e83 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:44.354458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227afaf-0692-4bb4-a733-b6a350ea2e83 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:44.354683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227afaf-0692-4bb4-a733-b6a350ea2e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:44.373072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f227afaf-0692-4bb4-a733-b6a350ea2e83 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:44.374298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f227afaf-0692-4bb4-a733-b6a350ea2e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:44.439400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 471/952] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:44.707520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f575-5cc6-40b5-88e8-641d85bd4fc0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:44.750217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:44.806365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f575-5cc6-40b5-88e8-641d85bd4fc0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:44.839200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:44.839924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:44.840209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:44.840483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:44.851234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:53:44.856040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:44.857126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:44.858192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 482/953] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 26 10:53:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:44.863331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227afaf-0692-4bb4-a733-b6a350ea2e83 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:44.867035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:44.959847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227afaf-0692-4bb4-a733-b6a350ea2e83 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:44.993094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:44.993693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:44.993974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:44.994250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:44.994897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:44.995198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:44.996590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:45.004611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 472/954] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 26 10:53:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:45.010553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:45.109067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:45.115654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4bda-161b-4cad-957b-e66d0fff652d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:45.144142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:45.144758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:45.145033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:45.145302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:45.156088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:45.157520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:45.171920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:45.179174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:45.180167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:45.180167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:45.180167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:45.180167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 483/955] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:45.180398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:45.277232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752449c-da1b-4b47-a9e0-7d66bfbff514 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:45.309743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:45.310341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:45.310615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:45.310881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:45.311504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:45.311821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:45.313180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:45.324928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 473/956] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:53:45.330655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:45.347496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:45.428711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63951568-01aa-4d07-b8da-4e208b4beb33 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:45.462865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:45.463482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:45.463758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:45.464061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:45.471285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:45.472743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 10:53:45.483942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:45.485026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 10:53:45.486186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:45.487931 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 484/957] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:45.491031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:45.588607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e60d9b-f947-466d-9b3a-1d7a7ffa75d2 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:45.613308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:45.622304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:45.622901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:45.623181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:45.623450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:45.624192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:45.624493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:45.626009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:45.643335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 474/958] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 26 10:53:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:53:45.649264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:45.664462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:45.745509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d8122-7719-45b1-a499-c4ca525e6861 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:45.778216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:45.778828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:45.779100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:45.779364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:45.845189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:45.846545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:45.859891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:45.944739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 485/959] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 26 10:53:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 10:53:45.950769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:46.050740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c650b-01bd-4d38-b9f7-3a4fe5128aff None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:46.085446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:46.086093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:46.086370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:46.086639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:46.096322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:46.097695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:53:46.112934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:46.114047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:53:46.115840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:53:46.115840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:53:46.115840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:46.115840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 475/960] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 10:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:46.120422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:46.216712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9d529-68d2-43df-a41c-6eb5834deba0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:46.230252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:46.251498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:46.252126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:46.252403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:46.252674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:46.253303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:46.253603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:46.255225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:46.266016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 486/961] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 26 10:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:46.271817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:46.307142 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:46.369924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d458-dfdc-4c7a-b4ce-c79987d256ab None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:46.403348 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:46.404119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:46.404398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:46.404670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:46.411057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:46.413732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 10:53:46.425818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:46.427560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 10:53:46.428772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 476/962] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 26 10:53:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:46.433112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:46.531807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd6e-78a3-4726-a47b-1a6392795d05 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:46.564646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c8c8b-41be-4871-9a32-2b97b47f1ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:46.565248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c8c8b-41be-4871-9a32-2b97b47f1ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:46.565524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c8c8b-41be-4871-9a32-2b97b47f1ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:46.565796 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c8c8b-41be-4871-9a32-2b97b47f1ef4 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:46.566472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c8c8b-41be-4871-9a32-2b97b47f1ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:46.566776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c8c8b-41be-4871-9a32-2b97b47f1ef4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:46.568257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c8c8b-41be-4871-9a32-2b97b47f1ef4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:46.579803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 487/963] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 26 10:53:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:46.586582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:46.681301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:46.686803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058d78-74bb-453a-b0f9-f18d16692594 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:46.720399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e58519-1c12-49ef-a4d6-b8c9d2dd17d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:46.720991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58519-1c12-49ef-a4d6-b8c9d2dd17d5 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:46.721264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58519-1c12-49ef-a4d6-b8c9d2dd17d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:46.721530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58519-1c12-49ef-a4d6-b8c9d2dd17d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:46.722233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e58519-1c12-49ef-a4d6-b8c9d2dd17d5 None admin] RBAC: Adding query filter params (role.id=25ab652070944ce094aac8e7e6ae0d77, scope.project.id=5968f35bae3d4643bc199a16ab96770b, user.id=4c154983aff14d9895fb4b542db61e79) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:46.722538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e58519-1c12-49ef-a4d6-b8c9d2dd17d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:46.723992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e58519-1c12-49ef-a4d6-b8c9d2dd17d5 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:46.730180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 477/964] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 26 10:53:46 2022] GET /identity/v3/role_assignments?user.id=4c154983aff14d9895fb4b542db61e79&scope.project.id=5968f35bae3d4643bc199a16ab96770b&role.id=25ab652070944ce094aac8e7e6ae0d77 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:48.986110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd70485-4d1d-4bb2-b8d9-99bd16dae718 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:48.986639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd70485-4d1d-4bb2-b8d9-99bd16dae718 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:48.986731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd70485-4d1d-4bb2-b8d9-99bd16dae718 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:48.988072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 488/965] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:48.994254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec00a6e-d760-42ee-ae50-841404fcfdba None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:48.994673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec00a6e-d760-42ee-ae50-841404fcfdba None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:48.994814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec00a6e-d760-42ee-ae50-841404fcfdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:49.014390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3ec00a6e-d760-42ee-ae50-841404fcfdba None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:49.015667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3ec00a6e-d760-42ee-ae50-841404fcfdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:49.090917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 478/966] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:49.372575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd70485-4d1d-4bb2-b8d9-99bd16dae718 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:49.404603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:49.476210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd70485-4d1d-4bb2-b8d9-99bd16dae718 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:49.510673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:49.511338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:49.511627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:49.511949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:49.512517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:49.513911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:49.523430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:49.524687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:49.525730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 489/967] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:53:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:49.530945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec00a6e-d760-42ee-ae50-841404fcfdba None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:49.622555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:49.632613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec00a6e-d760-42ee-ae50-841404fcfdba None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:49.668002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed8923-d759-46e4-b5d9-baa72fb4a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:49.668600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed8923-d759-46e4-b5d9-baa72fb4a59f None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:49.668877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed8923-d759-46e4-b5d9-baa72fb4a59f None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:49.669145 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed8923-d759-46e4-b5d9-baa72fb4a59f None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:49.669793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed8923-d759-46e4-b5d9-baa72fb4a59f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:49.670109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed8923-d759-46e4-b5d9-baa72fb4a59f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:49.671533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed8923-d759-46e4-b5d9-baa72fb4a59f None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:49.678232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 479/968] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 26 10:53:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:49.684116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:49.800140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:49.805894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4eff9a-4d57-4fd5-80da-aa8eab651a32 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:49.838715 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7d463-e687-4889-887c-62c473391c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:49.839370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d463-e687-4889-887c-62c473391c58 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:49.839644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d463-e687-4889-887c-62c473391c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:49.839942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d463-e687-4889-887c-62c473391c58 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:49.840561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7d463-e687-4889-887c-62c473391c58 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:49.840857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7d463-e687-4889-887c-62c473391c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:49.842208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7d463-e687-4889-887c-62c473391c58 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:49.846610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 490/969] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 26 10:53:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:52.108737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44cfc8-4045-41ce-9493-643b849fa804 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:52.109335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44cfc8-4045-41ce-9493-643b849fa804 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:52.109335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44cfc8-4045-41ce-9493-643b849fa804 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:52.110606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 480/970] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:52.116473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129e625-263c-4f3f-96ef-c6a12c81e8af None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:52.116900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129e625-263c-4f3f-96ef-c6a12c81e8af None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:52.117090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129e625-263c-4f3f-96ef-c6a12c81e8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:52.135612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8129e625-263c-4f3f-96ef-c6a12c81e8af None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:52.136838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8129e625-263c-4f3f-96ef-c6a12c81e8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:52.202503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 491/971] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:52.472711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44cfc8-4045-41ce-9493-643b849fa804 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:52.574859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44cfc8-4045-41ce-9493-643b849fa804 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:52.609467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91f97b-9ef4-4e8e-98d4-4001667cb062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:52.610068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f97b-9ef4-4e8e-98d4-4001667cb062 None admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:52.610344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f97b-9ef4-4e8e-98d4-4001667cb062 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:52.610615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f97b-9ef4-4e8e-98d4-4001667cb062 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:52.611262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91f97b-9ef4-4e8e-98d4-4001667cb062 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:52.612656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91f97b-9ef4-4e8e-98d4-4001667cb062 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:52.787382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 481/972] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 26 10:53:52 2022] POST /identity/v3/services => generated 246 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:53:54.947844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c69e3-4eda-4c5e-a05f-5eefd0f0d47f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:54.948433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c69e3-4eda-4c5e-a05f-5eefd0f0d47f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:54.948433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c69e3-4eda-4c5e-a05f-5eefd0f0d47f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:54.949737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 492/973] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:54.954983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d878b-f650-435f-9e4c-e29d04e2d01e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:54.968685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d878b-f650-435f-9e4c-e29d04e2d01e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:54.968685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d878b-f650-435f-9e4c-e29d04e2d01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:54.973413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-110d878b-f650-435f-9e4c-e29d04e2d01e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:54.975290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-110d878b-f650-435f-9e4c-e29d04e2d01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:54.984282 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:55.047213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 482/974] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:55.313067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c69e3-4eda-4c5e-a05f-5eefd0f0d47f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:55.403271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:55.413504 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c69e3-4eda-4c5e-a05f-5eefd0f0d47f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:55.447510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:55.448200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:55.448666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:55.449097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:55.450022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:55.451364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:55.460412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:55.462364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:55.463395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 493/975] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:53:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:53:55.467612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d878b-f650-435f-9e4c-e29d04e2d01e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:55.564963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d878b-f650-435f-9e4c-e29d04e2d01e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:55.600171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:55.600771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:55.601045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:55.601312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:55.601932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:55.602227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:55.603605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:55.608612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 483/976] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 26 10:53:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:53:55.615147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:55.712750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985af09-1508-46a5-8b12-a90ccb3ce007 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:55.745407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3729df9-2847-46b7-a18b-01f14e23e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:55.745998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729df9-2847-46b7-a18b-01f14e23e08e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:55.746274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729df9-2847-46b7-a18b-01f14e23e08e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:55.746544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729df9-2847-46b7-a18b-01f14e23e08e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:55.747220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3729df9-2847-46b7-a18b-01f14e23e08e None admin] RBAC: Adding query filter params (interface=public, service_id=75a8178d1b9a4c8aacab57eb78d14b6f, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:55.747522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3729df9-2847-46b7-a18b-01f14e23e08e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:55.748906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3729df9-2847-46b7-a18b-01f14e23e08e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:55.753928 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 494/977] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 26 10:53:55 2022] GET /identity/v3/endpoints?service_id=75a8178d1b9a4c8aacab57eb78d14b6f&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:55.759818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:55.763502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:55.856504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2652419-ebd5-4140-8b98-fe72b7287506 None admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:55.890703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e16dba-0fc3-47a9-bd07-9f8f8dcaa3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:55.891297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16dba-0fc3-47a9-bd07-9f8f8dcaa3af None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:55.891574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16dba-0fc3-47a9-bd07-9f8f8dcaa3af None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:55.891874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16dba-0fc3-47a9-bd07-9f8f8dcaa3af None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:55.892472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e16dba-0fc3-47a9-bd07-9f8f8dcaa3af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:55.893836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e16dba-0fc3-47a9-bd07-9f8f8dcaa3af None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:55.900435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 484/978] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 26 10:53:55 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 10:53:58.129808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac45dbe-50d1-4003-ae63-f9dcf8e66509 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:58.130378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac45dbe-50d1-4003-ae63-f9dcf8e66509 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:58.130378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac45dbe-50d1-4003-ae63-f9dcf8e66509 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:58.131713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 495/979] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 10:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:53:58.137499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab92566-ce11-45b4-8b90-9f0cbdddd98d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:58.137964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab92566-ce11-45b4-8b90-9f0cbdddd98d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:58.138425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab92566-ce11-45b4-8b90-9f0cbdddd98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:58.158090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6ab92566-ce11-45b4-8b90-9f0cbdddd98d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:53:58.159806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6ab92566-ce11-45b4-8b90-9f0cbdddd98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:53:58.228321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:58.233847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 485/980] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 10:53:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 10:53:58.507052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac45dbe-50d1-4003-ae63-f9dcf8e66509 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:58.526736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:58.608541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac45dbe-50d1-4003-ae63-f9dcf8e66509 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:58.640541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:58.641160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:58.641437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:58.641706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:58.642357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:58.643729 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:53:58.652839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:53:58.654027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 10:53:58.655053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 496/981] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 26 10:53:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:53:58.660021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab92566-ce11-45b4-8b90-9f0cbdddd98d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:58.759513 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab92566-ce11-45b4-8b90-9f0cbdddd98d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:58.793941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe62a24-da29-426c-880a-d35ead1120f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:58.794547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe62a24-da29-426c-880a-d35ead1120f3 None admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:58.794826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe62a24-da29-426c-880a-d35ead1120f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:58.795100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe62a24-da29-426c-880a-d35ead1120f3 None admin] PATH_INFO: `/v3/services` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:58.795715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe62a24-da29-426c-880a-d35ead1120f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:53:58.796043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe62a24-da29-426c-880a-d35ead1120f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:58.797404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe62a24-da29-426c-880a-d35ead1120f3 None admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:58.802495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 486/982] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 26 10:53:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:53:58.808515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:53:58.890060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:53:58.908499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fa46e-68f6-459d-94b1-81e573847968 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:53:58.941700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba28a4f-6fcc-4d0d-a5c0-a08936dbb99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:53:58.942943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba28a4f-6fcc-4d0d-a5c0-a08936dbb99e None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:53:58.943594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba28a4f-6fcc-4d0d-a5c0-a08936dbb99e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:53:58.943897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba28a4f-6fcc-4d0d-a5c0-a08936dbb99e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:53:58.944547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba28a4f-6fcc-4d0d-a5c0-a08936dbb99e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:53:58.945887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba28a4f-6fcc-4d0d-a5c0-a08936dbb99e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:53:59.090567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 497/983] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 10:53:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:54:08.799426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=69 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:54:09.085986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:45.895747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480d15d-68f0-4d89-9b85-aa6fac5ea3db None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:45.896232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480d15d-68f0-4d89-9b85-aa6fac5ea3db None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:45.896318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480d15d-68f0-4d89-9b85-aa6fac5ea3db None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:45.903057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69136d-2e23-42b1-9941-cffd85c6acf5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:45.903485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69136d-2e23-42b1-9941-cffd85c6acf5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:45.903605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69136d-2e23-42b1-9941-cffd85c6acf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:45.906651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 487/984] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:01:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:46.005400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-db69136d-2e23-42b1-9941-cffd85c6acf5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:46.007480 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-db69136d-2e23-42b1-9941-cffd85c6acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:46.211955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 498/985] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:01:46.519128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fea229-6725-4061-b075-69c33dd7043e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:46.519498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fea229-6725-4061-b075-69c33dd7043e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:46.519676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fea229-6725-4061-b075-69c33dd7043e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:46.521216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 488/986] 10.222.0.20 () {54 vars in 964 bytes} [Wed Oct 26 11:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:46.528303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b335535-2845-4482-86e9-c55a1fba2e78 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:46.528605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b335535-2845-4482-86e9-c55a1fba2e78 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:46.528884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b335535-2845-4482-86e9-c55a1fba2e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:46.631643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0b335535-2845-4482-86e9-c55a1fba2e78 None None] MFA Rules not processed for user `ce0cb1d976784ea6af783fd193ddac20`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:46.632895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0b335535-2845-4482-86e9-c55a1fba2e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:46.656115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:46.721958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 499/987] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Oct 26 11:01:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:01:46.732450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fea229-6725-4061-b075-69c33dd7043e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:46.734706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-64fea229-6725-4061-b075-69c33dd7043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:46.872616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fea229-6725-4061-b075-69c33dd7043e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:46.963683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe38db9-7c3a-460a-82f2-01c95ec2d59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:46.964942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe38db9-7c3a-460a-82f2-01c95ec2d59f service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:46.965221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe38db9-7c3a-460a-82f2-01c95ec2d59f service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:46.965490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe38db9-7c3a-460a-82f2-01c95ec2d59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:46.967101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5fe38db9-7c3a-460a-82f2-01c95ec2d59f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:47.028568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:47.098790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe38db9-7c3a-460a-82f2-01c95ec2d59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:47.100517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe38db9-7c3a-460a-82f2-01c95ec2d59f service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:47.121828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 489/988] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:01:49.605092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3eede8-a48e-4edd-8782-fa6e36a33bad None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:49.605641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3eede8-a48e-4edd-8782-fa6e36a33bad None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:49.605641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3eede8-a48e-4edd-8782-fa6e36a33bad None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:49.606989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 500/989] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:49.612186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b74168-784f-4d04-aa7b-e3a49b11da54 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:49.612606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b74168-784f-4d04-aa7b-e3a49b11da54 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:49.612720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b74168-784f-4d04-aa7b-e3a49b11da54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:49.687918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b1b74168-784f-4d04-aa7b-e3a49b11da54 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:49.689203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b1b74168-784f-4d04-aa7b-e3a49b11da54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:49.785127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 490/990] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:01:50.084104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3456d69-4995-4d61-ab33-51957b8cb8c8 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:50.084387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3456d69-4995-4d61-ab33-51957b8cb8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:50.084664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3456d69-4995-4d61-ab33-51957b8cb8c8 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:50.086030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 501/991] 10.222.0.20 () {54 vars in 964 bytes} [Wed Oct 26 11:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:50.092290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb435169-a449-4997-8b5a-e831da96bff9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:50.092577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb435169-a449-4997-8b5a-e831da96bff9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:50.092852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb435169-a449-4997-8b5a-e831da96bff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:50.104460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:50.168770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-bb435169-a449-4997-8b5a-e831da96bff9 None None] MFA Rules not processed for user `ce0cb1d976784ea6af783fd193ddac20`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:50.169972 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bb435169-a449-4997-8b5a-e831da96bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:50.244291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 491/992] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Oct 26 11:01:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:01:50.253427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3456d69-4995-4d61-ab33-51957b8cb8c8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:50.254941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e3456d69-4995-4d61-ab33-51957b8cb8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:50.356267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3456d69-4995-4d61-ab33-51957b8cb8c8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:50.425249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:50.427462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7e1ef-45c1-4d43-828c-3d0dd89b3583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:50.428094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7e1ef-45c1-4d43-828c-3d0dd89b3583 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:50.428373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7e1ef-45c1-4d43-828c-3d0dd89b3583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:50.428642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7e1ef-45c1-4d43-828c-3d0dd89b3583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:50.430203 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-68e7e1ef-45c1-4d43-828c-3d0dd89b3583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:50.535778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7e1ef-45c1-4d43-828c-3d0dd89b3583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:50.537223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7e1ef-45c1-4d43-828c-3d0dd89b3583 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:50.555266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 502/993] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:01:53.457835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1467-933f-4187-bf02-b8483db835af None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:53.458353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1467-933f-4187-bf02-b8483db835af None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:53.458353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a1467-933f-4187-bf02-b8483db835af None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:53.459714 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 492/994] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:53.464968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156993b7-d0ba-4e70-b8d8-6f985ebfba4c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:53.465407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156993b7-d0ba-4e70-b8d8-6f985ebfba4c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:53.465501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156993b7-d0ba-4e70-b8d8-6f985ebfba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:53.531190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-156993b7-d0ba-4e70-b8d8-6f985ebfba4c None None] MFA Rules not processed for user `85cda2bc952e4d8a99a088ee03cdeaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:53.532471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-156993b7-d0ba-4e70-b8d8-6f985ebfba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:53.583039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:53.627472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 503/995] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:01:53.911124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1467-933f-4187-bf02-b8483db835af None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:53.912708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-649a1467-933f-4187-bf02-b8483db835af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:54.016627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a1467-933f-4187-bf02-b8483db835af None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:54.019396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:54.091459 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ca37-d166-43d9-9441-aee75ca90274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:54.092093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ca37-d166-43d9-9441-aee75ca90274 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:54.092371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ca37-d166-43d9-9441-aee75ca90274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:54.092641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ca37-d166-43d9-9441-aee75ca90274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:54.094227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7be0ca37-d166-43d9-9441-aee75ca90274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:54.201320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0ca37-d166-43d9-9441-aee75ca90274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:54.202666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0ca37-d166-43d9-9441-aee75ca90274 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:54.221202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 493/996] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:01:57.157618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634bb3-0e36-4e50-a604-296c7a1cd2c6 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:57.158142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634bb3-0e36-4e50-a604-296c7a1cd2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:57.158142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634bb3-0e36-4e50-a604-296c7a1cd2c6 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:57.159484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 504/997] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:57.165305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453e1d7-20f5-413c-a1ef-f8d0ab904ce0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:57.165705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453e1d7-20f5-413c-a1ef-f8d0ab904ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:57.165857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453e1d7-20f5-413c-a1ef-f8d0ab904ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:57.206935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:57.232330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0453e1d7-20f5-413c-a1ef-f8d0ab904ce0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:57.233570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0453e1d7-20f5-413c-a1ef-f8d0ab904ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:57.329611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 494/998] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:01:57.618439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21634bb3-0e36-4e50-a604-296c7a1cd2c6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:57.620230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-21634bb3-0e36-4e50-a604-296c7a1cd2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:57.721438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21634bb3-0e36-4e50-a604-296c7a1cd2c6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:57.793167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0eab9-9583-4e44-b3b1-458ad974b4d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:57.793759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0eab9-9583-4e44-b3b1-458ad974b4d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:57.794035 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0eab9-9583-4e44-b3b1-458ad974b4d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:57.794301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0eab9-9583-4e44-b3b1-458ad974b4d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:57.795889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4af0eab9-9583-4e44-b3b1-458ad974b4d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:57.802509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:57.902004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0eab9-9583-4e44-b3b1-458ad974b4d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:57.903345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0eab9-9583-4e44-b3b1-458ad974b4d3 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:57.921161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 505/999] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:01:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:01.056737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d11c-8110-47fa-9128-63f3354341e0 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:01.057259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d11c-8110-47fa-9128-63f3354341e0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:01.057347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d11c-8110-47fa-9128-63f3354341e0 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:01.058624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 495/1000] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:01.063934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84babb-f7aa-4f24-ab80-5fa8ef144d39 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:01.064375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84babb-f7aa-4f24-ab80-5fa8ef144d39 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:01.064826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84babb-f7aa-4f24-ab80-5fa8ef144d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:01.132774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3b84babb-f7aa-4f24-ab80-5fa8ef144d39 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:01.134476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3b84babb-f7aa-4f24-ab80-5fa8ef144d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:01.193478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:01.238815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 506/1001] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:01.529851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43d11c-8110-47fa-9128-63f3354341e0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:01.531455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0d43d11c-8110-47fa-9128-63f3354341e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:01.636857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43d11c-8110-47fa-9128-63f3354341e0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:01.651361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:01.713892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc4dfc-2bd8-4a18-b189-126f2a65a22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:01.714485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc4dfc-2bd8-4a18-b189-126f2a65a22b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:01.714758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc4dfc-2bd8-4a18-b189-126f2a65a22b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:01.715023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc4dfc-2bd8-4a18-b189-126f2a65a22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:01.716621 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6bcc4dfc-2bd8-4a18-b189-126f2a65a22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:01.825499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc4dfc-2bd8-4a18-b189-126f2a65a22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:01.826934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc4dfc-2bd8-4a18-b189-126f2a65a22b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:01.845686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 496/1002] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:04.080602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc3ee3-95c7-4885-819b-c5e02697f122 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:04.081165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc3ee3-95c7-4885-819b-c5e02697f122 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:04.081165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc3ee3-95c7-4885-819b-c5e02697f122 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:04.082490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 507/1003] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:04.087813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e2b89-9e29-4fe1-9ddc-c472b790b19f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:04.088237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e2b89-9e29-4fe1-9ddc-c472b790b19f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:04.088381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e2b89-9e29-4fe1-9ddc-c472b790b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:04.143748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:04.161277 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-581e2b89-9e29-4fe1-9ddc-c472b790b19f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:04.162547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-581e2b89-9e29-4fe1-9ddc-c472b790b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:04.264441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 497/1004] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:04.563517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc3ee3-95c7-4885-819b-c5e02697f122 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:04.565167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-07bc3ee3-95c7-4885-819b-c5e02697f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:04.703972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc3ee3-95c7-4885-819b-c5e02697f122 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:04.797763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfd3f5-694f-44b2-868c-4f5028af572f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:04.798375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfd3f5-694f-44b2-868c-4f5028af572f service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:04.798655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfd3f5-694f-44b2-868c-4f5028af572f service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:04.798921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfd3f5-694f-44b2-868c-4f5028af572f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:04.800574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c4bfd3f5-694f-44b2-868c-4f5028af572f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:04.822186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:04.918516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfd3f5-694f-44b2-868c-4f5028af572f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:04.919986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfd3f5-694f-44b2-868c-4f5028af572f service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:04.938426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 508/1005] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:07.189162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d393d9-6058-4c8b-90da-49b2672c5993 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:07.189701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d393d9-6058-4c8b-90da-49b2672c5993 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:07.189701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d393d9-6058-4c8b-90da-49b2672c5993 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:07.191032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 498/1006] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:07.196850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2057dfc-2397-44a9-b1c7-e068c8fc5b4d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:07.197353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2057dfc-2397-44a9-b1c7-e068c8fc5b4d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:07.197450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2057dfc-2397-44a9-b1c7-e068c8fc5b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:07.264775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b2057dfc-2397-44a9-b1c7-e068c8fc5b4d None None] MFA Rules not processed for user `85cda2bc952e4d8a99a088ee03cdeaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:07.265977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b2057dfc-2397-44a9-b1c7-e068c8fc5b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:07.335755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:07.366278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 509/1007] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:07.653449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d393d9-6058-4c8b-90da-49b2672c5993 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:07.655105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-74d393d9-6058-4c8b-90da-49b2672c5993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:07.776041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d393d9-6058-4c8b-90da-49b2672c5993 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:07.799483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:07.862386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15bd14-9f76-440d-aab1-259e4b1c84b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:07.863241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15bd14-9f76-440d-aab1-259e4b1c84b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:07.863719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15bd14-9f76-440d-aab1-259e4b1c84b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:07.864210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15bd14-9f76-440d-aab1-259e4b1c84b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:07.866101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ea15bd14-9f76-440d-aab1-259e4b1c84b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:07.987850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15bd14-9f76-440d-aab1-259e4b1c84b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:07.989226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15bd14-9f76-440d-aab1-259e4b1c84b8 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:08.007881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 499/1008] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:10.543475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b45ba4-549d-4cd6-b985-a7b096603ac2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:10.544053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b45ba4-549d-4cd6-b985-a7b096603ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:10.544053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b45ba4-549d-4cd6-b985-a7b096603ac2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:10.545422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 510/1009] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:10.550711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57ecdd-8b26-47c9-9ed9-b263bf97f8af None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:10.551141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57ecdd-8b26-47c9-9ed9-b263bf97f8af None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:10.551264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57ecdd-8b26-47c9-9ed9-b263bf97f8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:10.616051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:10.619250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4d57ecdd-8b26-47c9-9ed9-b263bf97f8af None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:10.620551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4d57ecdd-8b26-47c9-9ed9-b263bf97f8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:10.714474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 500/1010] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:10.995985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b45ba4-549d-4cd6-b985-a7b096603ac2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:10.997530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-80b45ba4-549d-4cd6-b985-a7b096603ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:11.100912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b45ba4-549d-4cd6-b985-a7b096603ac2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:11.173180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5181d-2d04-477f-af33-6142523e945e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:11.173774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5181d-2d04-477f-af33-6142523e945e service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:11.174052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5181d-2d04-477f-af33-6142523e945e service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:11.174319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5181d-2d04-477f-af33-6142523e945e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:11.175935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-15b5181d-2d04-477f-af33-6142523e945e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:11.203197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:11.283223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5181d-2d04-477f-af33-6142523e945e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:11.284636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5181d-2d04-477f-af33-6142523e945e service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:11.302019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 511/1011] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:13.513951 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7329cb8-f0a0-48e1-b980-6cb3d2df09b4 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:13.514537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7329cb8-f0a0-48e1-b980-6cb3d2df09b4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:13.514537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7329cb8-f0a0-48e1-b980-6cb3d2df09b4 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:13.515853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 501/1012] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:13.521003 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e4ac-f890-4b5e-b2aa-01dd7c83cd70 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:13.521443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e4ac-f890-4b5e-b2aa-01dd7c83cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:13.521537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e4ac-f890-4b5e-b2aa-01dd7c83cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:13.587000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7a09e4ac-f890-4b5e-b2aa-01dd7c83cd70 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:13.588221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7a09e4ac-f890-4b5e-b2aa-01dd7c83cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:13.659928 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:13.683036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 512/1013] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:13.971070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7329cb8-f0a0-48e1-b980-6cb3d2df09b4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:13.972700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f7329cb8-f0a0-48e1-b980-6cb3d2df09b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:14.076511 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7329cb8-f0a0-48e1-b980-6cb3d2df09b4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:14.105085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:14.149718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb360a07-b6a2-435e-83ef-127b6bdda964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:14.150313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb360a07-b6a2-435e-83ef-127b6bdda964 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:14.150593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb360a07-b6a2-435e-83ef-127b6bdda964 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:14.150859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb360a07-b6a2-435e-83ef-127b6bdda964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:14.152465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bb360a07-b6a2-435e-83ef-127b6bdda964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:14.259498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb360a07-b6a2-435e-83ef-127b6bdda964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:14.260881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb360a07-b6a2-435e-83ef-127b6bdda964 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:14.279499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 502/1014] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:17.041767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b7ae2-cce0-44a7-aa0f-37c5e5d9492c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:17.042199 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b7ae2-cce0-44a7-aa0f-37c5e5d9492c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:17.042294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b7ae2-cce0-44a7-aa0f-37c5e5d9492c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:17.043814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 513/1015] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:17.049048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ac272-24bb-4659-9a3d-751ac07acad8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:17.049423 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ac272-24bb-4659-9a3d-751ac07acad8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:17.049587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ac272-24bb-4659-9a3d-751ac07acad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:17.114718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b42ac272-24bb-4659-9a3d-751ac07acad8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:17.116005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b42ac272-24bb-4659-9a3d-751ac07acad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:17.124151 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:17.210617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 503/1016] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:02:17.492905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b7ae2-cce0-44a7-aa0f-37c5e5d9492c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:17.494438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-617b7ae2-cce0-44a7-aa0f-37c5e5d9492c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:17.594789 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b7ae2-cce0-44a7-aa0f-37c5e5d9492c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:17.666838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af086982-d1f4-4ff8-a551-172ecb84e314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:17.667231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af086982-d1f4-4ff8-a551-172ecb84e314 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:17.667510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af086982-d1f4-4ff8-a551-172ecb84e314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:17.667780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af086982-d1f4-4ff8-a551-172ecb84e314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:17.669383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-af086982-d1f4-4ff8-a551-172ecb84e314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:17.705074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:17.777098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af086982-d1f4-4ff8-a551-172ecb84e314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:17.778451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af086982-d1f4-4ff8-a551-172ecb84e314 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:17.796355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 514/1017] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:22.115033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7ddab-78a7-4759-8f7e-5e3976c7f0e6 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:22.115581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7ddab-78a7-4759-8f7e-5e3976c7f0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:22.115581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7ddab-78a7-4759-8f7e-5e3976c7f0e6 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:22.117616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 504/1018] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:22.123160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d36118-b405-4d84-892e-4bd743bcf6e4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:22.123550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d36118-b405-4d84-892e-4bd743bcf6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:22.123740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d36118-b405-4d84-892e-4bd743bcf6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:22.190116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b0d36118-b405-4d84-892e-4bd743bcf6e4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:22.191245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b0d36118-b405-4d84-892e-4bd743bcf6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:22.278096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:22.289567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 515/1019] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:22.572705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7ddab-78a7-4759-8f7e-5e3976c7f0e6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:22.574243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-adf7ddab-78a7-4759-8f7e-5e3976c7f0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:22.677977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7ddab-78a7-4759-8f7e-5e3976c7f0e6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:22.716360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:22.752574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c81f8-bc9a-4960-84b9-daa22f091438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:22.753178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c81f8-bc9a-4960-84b9-daa22f091438 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:22.753456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c81f8-bc9a-4960-84b9-daa22f091438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:22.753727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c81f8-bc9a-4960-84b9-daa22f091438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:22.755309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-138c81f8-bc9a-4960-84b9-daa22f091438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:22.867271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c81f8-bc9a-4960-84b9-daa22f091438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:22.868909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c81f8-bc9a-4960-84b9-daa22f091438 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:22.887598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 505/1020] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:26.006990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044dbf0-6e41-4064-9ea0-0d5c8a16c69a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:26.007572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044dbf0-6e41-4064-9ea0-0d5c8a16c69a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:26.007572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044dbf0-6e41-4064-9ea0-0d5c8a16c69a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:26.008841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 516/1021] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:26.014570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49939a5-6c54-4ea1-9893-9507916a10a2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:26.014983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49939a5-6c54-4ea1-9893-9507916a10a2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:26.015136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49939a5-6c54-4ea1-9893-9507916a10a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:26.082945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d49939a5-6c54-4ea1-9893-9507916a10a2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:26.084193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d49939a5-6c54-4ea1-9893-9507916a10a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:26.103364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:26.183979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 506/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:26.480400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044dbf0-6e41-4064-9ea0-0d5c8a16c69a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:26.482119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a044dbf0-6e41-4064-9ea0-0d5c8a16c69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:26.590605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044dbf0-6e41-4064-9ea0-0d5c8a16c69a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:26.665141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc388a7-5d16-4a7b-9695-e7083aea4d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:26.665741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc388a7-5d16-4a7b-9695-e7083aea4d1a service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:26.666027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc388a7-5d16-4a7b-9695-e7083aea4d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:26.666299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc388a7-5d16-4a7b-9695-e7083aea4d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:26.667921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-cbc388a7-5d16-4a7b-9695-e7083aea4d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:26.732441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:26.806663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc388a7-5d16-4a7b-9695-e7083aea4d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:26.808187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc388a7-5d16-4a7b-9695-e7083aea4d1a service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:26.828754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 517/1023] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:31.145978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7636e-32ef-476f-a0ed-825c1bb0978c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:31.146496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7636e-32ef-476f-a0ed-825c1bb0978c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:31.146593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7636e-32ef-476f-a0ed-825c1bb0978c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:31.148083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 507/1024] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:02:31.153632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fbec4-0dc7-4570-8f1c-8dfbc98cb8ca None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:31.154072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fbec4-0dc7-4570-8f1c-8dfbc98cb8ca None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:31.154375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fbec4-0dc7-4570-8f1c-8dfbc98cb8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:31.224941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-be7fbec4-0dc7-4570-8f1c-8dfbc98cb8ca None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:31.226157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-be7fbec4-0dc7-4570-8f1c-8dfbc98cb8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:31.328398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:31.330701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 518/1025] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:31.618942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7636e-32ef-476f-a0ed-825c1bb0978c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:31.620541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-52f7636e-32ef-476f-a0ed-825c1bb0978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:31.727688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7636e-32ef-476f-a0ed-825c1bb0978c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:31.778409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:31.805761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eabf9-9c6f-40d6-8fbc-83b01f3b2ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:31.806377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eabf9-9c6f-40d6-8fbc-83b01f3b2ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:31.806658 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eabf9-9c6f-40d6-8fbc-83b01f3b2ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:31.806932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eabf9-9c6f-40d6-8fbc-83b01f3b2ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:31.808579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2c2eabf9-9c6f-40d6-8fbc-83b01f3b2ed0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:31.922261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eabf9-9c6f-40d6-8fbc-83b01f3b2ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:31.923716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eabf9-9c6f-40d6-8fbc-83b01f3b2ed0 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:31.942794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 508/1026] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:36.968443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0740b-1525-41ff-bc31-af096156b1e2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:36.968443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0740b-1525-41ff-bc31-af096156b1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:36.969088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0740b-1525-41ff-bc31-af096156b1e2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:36.977024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 519/1027] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 26 11:02:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:36.977819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ed802-7589-438d-bfa4-470f586acde1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:36.978311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ed802-7589-438d-bfa4-470f586acde1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:36.978682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ed802-7589-438d-bfa4-470f586acde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:37.122523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3f9ed802-7589-438d-bfa4-470f586acde1 None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:37.124031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3f9ed802-7589-438d-bfa4-470f586acde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:37.145963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:37.248252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 509/1028] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 26 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:41.246969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf2f2a-e503-4e5c-978d-22e414175a4a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:41.247872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf2f2a-e503-4e5c-978d-22e414175a4a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:41.248309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf2f2a-e503-4e5c-978d-22e414175a4a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:41.249817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 520/1029] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 26 11:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:41.254836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20111c57-0e09-41ee-b9f0-1ba9c9e46ba6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:41.255237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20111c57-0e09-41ee-b9f0-1ba9c9e46ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:41.255398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20111c57-0e09-41ee-b9f0-1ba9c9e46ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:41.334646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-20111c57-0e09-41ee-b9f0-1ba9c9e46ba6 None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:41.335925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-20111c57-0e09-41ee-b9f0-1ba9c9e46ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:41.411354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:41.447737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 510/1030] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 26 11:02:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:42.977854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70e3ab-52cd-477f-b826-133c83b65583 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:42.978384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70e3ab-52cd-477f-b826-133c83b65583 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:42.978384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70e3ab-52cd-477f-b826-133c83b65583 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:42.979985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 521/1031] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 26 11:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:42.985492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728017-fcb2-47e5-b669-2641366b9bfa None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:42.985920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728017-fcb2-47e5-b669-2641366b9bfa None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:42.986045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728017-fcb2-47e5-b669-2641366b9bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:43.069868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-24728017-fcb2-47e5-b669-2641366b9bfa None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:43.071096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-24728017-fcb2-47e5-b669-2641366b9bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:43.179427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 511/1032] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 26 11:02:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:46.732715 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b458f3-1b20-4066-9e53-9aa49f950164 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:46.733259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b458f3-1b20-4066-9e53-9aa49f950164 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:46.733259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b458f3-1b20-4066-9e53-9aa49f950164 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:46.734516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 522/1033] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:46.739985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061be3a5-1c7a-4961-95f6-4d99d438a6d4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:46.740386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061be3a5-1c7a-4961-95f6-4d99d438a6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:46.740542 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061be3a5-1c7a-4961-95f6-4d99d438a6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:46.779670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:46.814327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-061be3a5-1c7a-4961-95f6-4d99d438a6d4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:46.815526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-061be3a5-1c7a-4961-95f6-4d99d438a6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:46.920845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 512/1034] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:47.222043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46543-e059-4203-bbe8-41dd2973098d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:47.222728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46543-e059-4203-bbe8-41dd2973098d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:47.223846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46543-e059-4203-bbe8-41dd2973098d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:47.227503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 523/1035] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 26 11:02:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:47.233874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7d5cb-ad95-4bbf-a9cb-4600a9b5e457 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:47.237887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7d5cb-ad95-4bbf-a9cb-4600a9b5e457 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:47.237887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7d5cb-ad95-4bbf-a9cb-4600a9b5e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:47.326432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c2d7d5cb-ad95-4bbf-a9cb-4600a9b5e457 None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:47.327631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c2d7d5cb-ad95-4bbf-a9cb-4600a9b5e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:47.381009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:47.452825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 513/1036] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 26 11:02:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:47.460973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46543-e059-4203-bbe8-41dd2973098d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:47.462474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a0c46543-e059-4203-bbe8-41dd2973098d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:47.622157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46543-e059-4203-bbe8-41dd2973098d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:47.701837 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:47.742683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97081e97-449a-43bc-997a-6b50e4c86a0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:47.743496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97081e97-449a-43bc-997a-6b50e4c86a0e service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:47.743740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97081e97-449a-43bc-997a-6b50e4c86a0e service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:47.743975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97081e97-449a-43bc-997a-6b50e4c86a0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:47.746103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-97081e97-449a-43bc-997a-6b50e4c86a0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:47.852316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97081e97-449a-43bc-997a-6b50e4c86a0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:47.853991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97081e97-449a-43bc-997a-6b50e4c86a0e service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:47.892004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 524/1037] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:50.297925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09bd1d-a252-459b-8386-dde222425822 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:50.297925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09bd1d-a252-459b-8386-dde222425822 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:50.298552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09bd1d-a252-459b-8386-dde222425822 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:50.304739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 514/1038] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:50.305764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1d287-7b4b-4497-b142-00cb9dfacb7a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:50.307051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1d287-7b4b-4497-b142-00cb9dfacb7a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:50.307051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1d287-7b4b-4497-b142-00cb9dfacb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:50.378973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-eba1d287-7b4b-4497-b142-00cb9dfacb7a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:50.387112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-eba1d287-7b4b-4497-b142-00cb9dfacb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:50.424280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:50.489755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 525/1039] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:02:50.795964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09bd1d-a252-459b-8386-dde222425822 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:50.797617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cd09bd1d-a252-459b-8386-dde222425822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:50.977346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09bd1d-a252-459b-8386-dde222425822 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:50.988292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:51.111072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7d655-9427-43d8-83e5-01ebbe909821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:51.111674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7d655-9427-43d8-83e5-01ebbe909821 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:51.111983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7d655-9427-43d8-83e5-01ebbe909821 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:51.112250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7d655-9427-43d8-83e5-01ebbe909821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:51.113828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-63e7d655-9427-43d8-83e5-01ebbe909821 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:51.208975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7d655-9427-43d8-83e5-01ebbe909821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:51.210840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7d655-9427-43d8-83e5-01ebbe909821 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:51.230339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 515/1040] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:53.894179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aeca9-d7b2-4238-a7be-2592817402c0 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:53.894707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aeca9-d7b2-4238-a7be-2592817402c0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:53.894707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40aeca9-d7b2-4238-a7be-2592817402c0 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:53.899828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 526/1041] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:53.902305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae140d9-21a6-4d0c-886d-68403a1e3cf0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:53.902737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae140d9-21a6-4d0c-886d-68403a1e3cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:53.903045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae140d9-21a6-4d0c-886d-68403a1e3cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:53.923196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:53.980114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7ae140d9-21a6-4d0c-886d-68403a1e3cf0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:53.981244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7ae140d9-21a6-4d0c-886d-68403a1e3cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:54.094458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 516/1042] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:54.392384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ca887-fce0-438e-a66b-eaaa79ed7cfb None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:54.392695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ca887-fce0-438e-a66b-eaaa79ed7cfb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:54.392933 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ca887-fce0-438e-a66b-eaaa79ed7cfb None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:54.394625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 527/1043] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 26 11:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:54.400711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6af67-e7c8-4965-a572-683be590ed3f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:54.401092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6af67-e7c8-4965-a572-683be590ed3f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:54.401439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6af67-e7c8-4965-a572-683be590ed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:54.481697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-dec6af67-e7c8-4965-a572-683be590ed3f None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:54.482909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-dec6af67-e7c8-4965-a572-683be590ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:54.520787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:54.609291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 517/1044] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 26 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:54.617951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ca887-fce0-438e-a66b-eaaa79ed7cfb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:54.619570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-811ca887-fce0-438e-a66b-eaaa79ed7cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:54.780285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ca887-fce0-438e-a66b-eaaa79ed7cfb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:54.803899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:54.908343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e8002-b071-464f-a00e-db6cc337e1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:54.909034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8002-b071-464f-a00e-db6cc337e1c4 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:54.909201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8002-b071-464f-a00e-db6cc337e1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:54.909470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8002-b071-464f-a00e-db6cc337e1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:54.911039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-902e8002-b071-464f-a00e-db6cc337e1c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:54.922283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad70a9-75cf-409a-8923-0ce607cff4e7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:54.922841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad70a9-75cf-409a-8923-0ce607cff4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:54.922841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad70a9-75cf-409a-8923-0ce607cff4e7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:54.924204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 518/1045] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:02:54.930031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac96e9-b233-46bd-80e9-84ca6166cbc0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:54.930429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac96e9-b233-46bd-80e9-84ca6166cbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:54.930796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac96e9-b233-46bd-80e9-84ca6166cbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:55.004704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e8002-b071-464f-a00e-db6cc337e1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:55.006851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e8002-b071-464f-a00e-db6cc337e1c4 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:55.007603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-55ac96e9-b233-46bd-80e9-84ca6166cbc0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:55.008846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-55ac96e9-b233-46bd-80e9-84ca6166cbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:55.027337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 528/1046] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:55.109943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:55.118839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 519/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:55.472726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b40f3f-1209-4441-8eb3-7aeb66687060 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:55.473101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b40f3f-1209-4441-8eb3-7aeb66687060 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:55.473450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b40f3f-1209-4441-8eb3-7aeb66687060 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:55.475835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 529/1048] 10.222.0.20 () {54 vars in 963 bytes} [Wed Oct 26 11:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:55.481067 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bb08d-ba48-4522-957b-8535174d3df7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:55.481356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bb08d-ba48-4522-957b-8535174d3df7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:55.481634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bb08d-ba48-4522-957b-8535174d3df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:55.564052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-830bb08d-ba48-4522-957b-8535174d3df7 None None] MFA Rules not processed for user `37b474a596cf42efb5e0e5a2a179735e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:55.565174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-830bb08d-ba48-4522-957b-8535174d3df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:55.646413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 520/1049] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Oct 26 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:55.654151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b40f3f-1209-4441-8eb3-7aeb66687060 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:55.655657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-07b40f3f-1209-4441-8eb3-7aeb66687060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:55.688019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:55.771343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b40f3f-1209-4441-8eb3-7aeb66687060 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:55.850893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307ae50-bcc8-4be0-bee9-19a828dd9f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:55.851437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307ae50-bcc8-4be0-bee9-19a828dd9f7e service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:55.851718 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307ae50-bcc8-4be0-bee9-19a828dd9f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:55.852017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307ae50-bcc8-4be0-bee9-19a828dd9f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:55.853654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1307ae50-bcc8-4be0-bee9-19a828dd9f7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:55.943011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:55.977606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307ae50-bcc8-4be0-bee9-19a828dd9f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:55.979043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307ae50-bcc8-4be0-bee9-19a828dd9f7e service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:55.997810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 530/1050] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:02:57.438710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c936-83dd-4beb-bb06-536754e74730 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:57.439303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c936-83dd-4beb-bb06-536754e74730 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:57.439303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7c936-83dd-4beb-bb06-536754e74730 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:57.440748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 521/1051] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:57.446581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86a2d1-5a63-455d-b47f-eb60e9e4f1ab None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:57.447086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86a2d1-5a63-455d-b47f-eb60e9e4f1ab None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:57.447365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86a2d1-5a63-455d-b47f-eb60e9e4f1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:57.515515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3a86a2d1-5a63-455d-b47f-eb60e9e4f1ab None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:57.516831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3a86a2d1-5a63-455d-b47f-eb60e9e4f1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:57.621816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 531/1052] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:02:57.919961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7c936-83dd-4beb-bb06-536754e74730 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:57.921578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-57b7c936-83dd-4beb-bb06-536754e74730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:58.014056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:58.083599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7c936-83dd-4beb-bb06-536754e74730 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:58.210150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e965a-e3fe-4ec3-ab1e-bd343eb18e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:58.210857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e965a-e3fe-4ec3-ab1e-bd343eb18e69 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:58.211138 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e965a-e3fe-4ec3-ab1e-bd343eb18e69 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:58.211415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e965a-e3fe-4ec3-ab1e-bd343eb18e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:58.213078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6b5e965a-e3fe-4ec3-ab1e-bd343eb18e69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:58.268096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:58.312992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e965a-e3fe-4ec3-ab1e-bd343eb18e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:58.314667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e965a-e3fe-4ec3-ab1e-bd343eb18e69 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:58.333413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 522/1053] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:00.687279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac3c4-2ce7-4f8c-845d-e226a91fbcfa None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:00.688080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac3c4-2ce7-4f8c-845d-e226a91fbcfa None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:00.688080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac3c4-2ce7-4f8c-845d-e226a91fbcfa None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:00.689224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 532/1054] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:00.694557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ce77-372e-44fd-95a7-5a5dc553db55 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:00.694964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ce77-372e-44fd-95a7-5a5dc553db55 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:00.695133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ce77-372e-44fd-95a7-5a5dc553db55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:00.764130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c9f4ce77-372e-44fd-95a7-5a5dc553db55 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:00.765337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c9f4ce77-372e-44fd-95a7-5a5dc553db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:00.866347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 523/1055] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:01.153187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bac3c4-2ce7-4f8c-845d-e226a91fbcfa None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:01.154829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f6bac3c4-2ce7-4f8c-845d-e226a91fbcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:01.196693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:01.315433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bac3c4-2ce7-4f8c-845d-e226a91fbcfa None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:01.437907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:01.444206 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892934e-4cbc-46a8-bc67-14f231045d08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:01.444881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892934e-4cbc-46a8-bc67-14f231045d08 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:01.445217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892934e-4cbc-46a8-bc67-14f231045d08 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:01.445544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892934e-4cbc-46a8-bc67-14f231045d08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:01.447208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c892934e-4cbc-46a8-bc67-14f231045d08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:01.547057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892934e-4cbc-46a8-bc67-14f231045d08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:01.548761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892934e-4cbc-46a8-bc67-14f231045d08 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:01.572046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 533/1056] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:03.967816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650050-b61c-4147-ac3b-fa8e14309ce3 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:03.968329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650050-b61c-4147-ac3b-fa8e14309ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:03.968329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650050-b61c-4147-ac3b-fa8e14309ce3 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:03.969681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 524/1057] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:03.975115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182677b-963c-4e1c-87ec-c60f6f034ec9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:03.975529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182677b-963c-4e1c-87ec-c60f6f034ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:03.975661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182677b-963c-4e1c-87ec-c60f6f034ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:04.042702 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f182677b-963c-4e1c-87ec-c60f6f034ec9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:04.043944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f182677b-963c-4e1c-87ec-c60f6f034ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:04.114914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:04.149420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 534/1058] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:04.441353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650050-b61c-4147-ac3b-fa8e14309ce3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:04.442913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-07650050-b61c-4147-ac3b-fa8e14309ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:04.472540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:04.592755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650050-b61c-4147-ac3b-fa8e14309ce3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:04.690807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:04.706958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db9ac1-d689-4390-9269-34c1f8aa8b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:04.707569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db9ac1-d689-4390-9269-34c1f8aa8b35 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:04.707889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db9ac1-d689-4390-9269-34c1f8aa8b35 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:04.708166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db9ac1-d689-4390-9269-34c1f8aa8b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:04.710096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b1db9ac1-d689-4390-9269-34c1f8aa8b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:04.803706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db9ac1-d689-4390-9269-34c1f8aa8b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:04.805467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db9ac1-d689-4390-9269-34c1f8aa8b35 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:04.824286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 525/1059] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:07.222766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7bbfb-c9b5-4d91-92d3-dc309bacb006 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:07.223351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7bbfb-c9b5-4d91-92d3-dc309bacb006 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:07.223351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7bbfb-c9b5-4d91-92d3-dc309bacb006 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:07.225962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 535/1060] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:07.232290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a0d-6939-4af8-979b-21cb87004f94 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:07.232802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a0d-6939-4af8-979b-21cb87004f94 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:07.233295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a0d-6939-4af8-979b-21cb87004f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:07.306016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-839c1a0d-6939-4af8-979b-21cb87004f94 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:07.307149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-839c1a0d-6939-4af8-979b-21cb87004f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:07.367677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:07.411377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 526/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:03:07.699556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7bbfb-c9b5-4d91-92d3-dc309bacb006 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:07.701211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9cc7bbfb-c9b5-4d91-92d3-dc309bacb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:07.852440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7bbfb-c9b5-4d91-92d3-dc309bacb006 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:07.906682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:07.968025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f475f-72ad-4ccf-a62e-71756b7b02df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:07.968632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f475f-72ad-4ccf-a62e-71756b7b02df service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:07.968914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f475f-72ad-4ccf-a62e-71756b7b02df service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:07.969184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f475f-72ad-4ccf-a62e-71756b7b02df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:07.970827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0a0f475f-72ad-4ccf-a62e-71756b7b02df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:08.064612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f475f-72ad-4ccf-a62e-71756b7b02df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:08.066203 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f475f-72ad-4ccf-a62e-71756b7b02df service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:08.085650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 536/1062] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:10.330799 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207622c6-7759-43d3-860f-b30187a6dfd1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:10.331479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207622c6-7759-43d3-860f-b30187a6dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:10.331479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207622c6-7759-43d3-860f-b30187a6dfd1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:10.332751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 527/1063] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:10.338333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4f0b-0b7f-40dc-898f-3d1bf4b5cf3b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:10.338805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4f0b-0b7f-40dc-898f-3d1bf4b5cf3b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:10.338805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c4f0b-0b7f-40dc-898f-3d1bf4b5cf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:10.407587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3b5c4f0b-0b7f-40dc-898f-3d1bf4b5cf3b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:10.408926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3b5c4f0b-0b7f-40dc-898f-3d1bf4b5cf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:10.418292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:10.512188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 537/1064] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:10.817199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207622c6-7759-43d3-860f-b30187a6dfd1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:10.819768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-207622c6-7759-43d3-860f-b30187a6dfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:10.979357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207622c6-7759-43d3-860f-b30187a6dfd1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:11.034965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:11.106154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018713d7-ba33-4990-990f-cbb86ccff92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:11.106912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018713d7-ba33-4990-990f-cbb86ccff92d service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:11.107306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018713d7-ba33-4990-990f-cbb86ccff92d service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:11.107659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018713d7-ba33-4990-990f-cbb86ccff92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:11.109726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-018713d7-ba33-4990-990f-cbb86ccff92d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:11.215265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018713d7-ba33-4990-990f-cbb86ccff92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:11.216897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018713d7-ba33-4990-990f-cbb86ccff92d service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:11.234982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 528/1065] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:13.672680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710a26-07a8-4084-9a01-d1e60792c8f9 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:13.673222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710a26-07a8-4084-9a01-d1e60792c8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:13.673222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710a26-07a8-4084-9a01-d1e60792c8f9 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:13.674720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 538/1066] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:13.680053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4ab7-52b5-4591-adf9-08b7c58abb6e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:13.680507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4ab7-52b5-4591-adf9-08b7c58abb6e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:13.680617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4ab7-52b5-4591-adf9-08b7c58abb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:13.747243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:13.749850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7c5b4ab7-52b5-4591-adf9-08b7c58abb6e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:13.751043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7c5b4ab7-52b5-4591-adf9-08b7c58abb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:13.853778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 529/1067] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:14.159294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7710a26-07a8-4084-9a01-d1e60792c8f9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:14.160997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c7710a26-07a8-4084-9a01-d1e60792c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:14.303835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:14.310666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7710a26-07a8-4084-9a01-d1e60792c8f9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:14.424172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bcb34-9e14-4e2f-aa34-ff607f9f949f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:14.425198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bcb34-9e14-4e2f-aa34-ff607f9f949f service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:14.425705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bcb34-9e14-4e2f-aa34-ff607f9f949f service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:14.426622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bcb34-9e14-4e2f-aa34-ff607f9f949f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:14.428861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7b4bcb34-9e14-4e2f-aa34-ff607f9f949f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:14.521311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bcb34-9e14-4e2f-aa34-ff607f9f949f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:14.522894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bcb34-9e14-4e2f-aa34-ff607f9f949f service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:14.537023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:14.541454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 539/1068] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:15.256863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980b61-9f84-489d-8df0-5763b7565fbe None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:15.257429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980b61-9f84-489d-8df0-5763b7565fbe None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:15.257429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980b61-9f84-489d-8df0-5763b7565fbe None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:15.258793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 530/1069] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:15.264542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b48c65-b075-432e-81c1-0ecdf5f90a2c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:15.264978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b48c65-b075-432e-81c1-0ecdf5f90a2c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:15.265119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b48c65-b075-432e-81c1-0ecdf5f90a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:15.332804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f2b48c65-b075-432e-81c1-0ecdf5f90a2c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:15.334614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f2b48c65-b075-432e-81c1-0ecdf5f90a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:15.432084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 540/1070] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:15.738091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fab4-c15f-4985-877b-a9c712560ae5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:15.738375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fab4-c15f-4985-877b-a9c712560ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:15.738646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648fab4-c15f-4985-877b-a9c712560ae5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:15.740630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 531/1071] 10.222.0.20 () {54 vars in 962 bytes} [Wed Oct 26 11:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:15.745952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339d76-3f96-4584-9c02-ee29371fef00 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:15.746457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339d76-3f96-4584-9c02-ee29371fef00 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:15.746731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed339d76-3f96-4584-9c02-ee29371fef00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:15.823593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ed339d76-3f96-4584-9c02-ee29371fef00 None None] MFA Rules not processed for user `9627e6a9e86e43b694f723bd4109460e`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:15.824857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ed339d76-3f96-4584-9c02-ee29371fef00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:15.844995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:15.907073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 541/1072] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Oct 26 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:15.915370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fab4-c15f-4985-877b-a9c712560ae5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:15.917023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d648fab4-c15f-4985-877b-a9c712560ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:16.020589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648fab4-c15f-4985-877b-a9c712560ae5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:16.051603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:16.098235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4b4ac-015b-4d93-9b8c-dd25a9cf0b9a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9627e6a9e86e43b694f723bd4109460e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:16.099278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b4ac-015b-4d93-9b8c-dd25a9cf0b9a service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:16.099562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b4ac-015b-4d93-9b8c-dd25a9cf0b9a service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:16.099869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b4ac-015b-4d93-9b8c-dd25a9cf0b9a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:16.101473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e3b4b4ac-015b-4d93-9b8c-dd25a9cf0b9a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:16.213928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4b4ac-015b-4d93-9b8c-dd25a9cf0b9a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:16.215376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4b4ac-015b-4d93-9b8c-dd25a9cf0b9a service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:16.234384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 532/1073] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:03:17.171007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4832f61-7599-47b3-b45d-260371ce09bb None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:17.171568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4832f61-7599-47b3-b45d-260371ce09bb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:17.171568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4832f61-7599-47b3-b45d-260371ce09bb None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:17.172943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 542/1074] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:17.179062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a55d72-f9ff-4d3d-9c61-12ef60c8a052 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:17.182889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a55d72-f9ff-4d3d-9c61-12ef60c8a052 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:17.182889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a55d72-f9ff-4d3d-9c61-12ef60c8a052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:17.198833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce690d-6461-4748-8160-21fa7b7444bc None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:17.199119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce690d-6461-4748-8160-21fa7b7444bc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:17.199420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce690d-6461-4748-8160-21fa7b7444bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:17.250523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-92a55d72-f9ff-4d3d-9c61-12ef60c8a052 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:17.251832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-92a55d72-f9ff-4d3d-9c61-12ef60c8a052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:17.261095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:17.270427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e5ce690d-6461-4748-8160-21fa7b7444bc None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:17.272730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e5ce690d-6461-4748-8160-21fa7b7444bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:17.354658 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 533/1075] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:17.360653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 543/1076] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:03:17.405286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e0b08-6a54-46f3-a596-a71b44bb4207 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:17.405572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e0b08-6a54-46f3-a596-a71b44bb4207 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:17.405845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e0b08-6a54-46f3-a596-a71b44bb4207 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:17.407194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 534/1077] 10.222.0.20 () {54 vars in 943 bytes} [Wed Oct 26 11:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:17.413811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:17.414100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:17.414376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:17.456655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:17.492585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] MFA Rules not processed for user `6bb4908b26e64c538c2b573f3cf26bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:17.493770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:17.600037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 544/1078] 10.222.0.20 () {58 vars in 1040 bytes} [Wed Oct 26 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:17.609040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e0b08-6a54-46f3-a596-a71b44bb4207 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:17.610627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b5e0b08-6a54-46f3-a596-a71b44bb4207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:17.654521 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:17.655722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:17.751407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e0b08-6a54-46f3-a596-a71b44bb4207 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:17.753480 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:17.755907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:17.757890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adfa59-4706-4d48-bf51-a2abbf882771 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:17.858403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ed5e4-fc3c-4481-9c84-17dc3e060c95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:17.858921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed5e4-fc3c-4481-9c84-17dc3e060c95 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:17.859200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed5e4-fc3c-4481-9c84-17dc3e060c95 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:17.859468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed5e4-fc3c-4481-9c84-17dc3e060c95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:17.861151 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c15ed5e4-fc3c-4481-9c84-17dc3e060c95 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:17.876915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86226c6-2ea6-4126-9f5c-dd0f7c53422f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:17.877620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86226c6-2ea6-4126-9f5c-dd0f7c53422f service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:17.877855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86226c6-2ea6-4126-9f5c-dd0f7c53422f service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:17.878130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86226c6-2ea6-4126-9f5c-dd0f7c53422f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:17.879741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c86226c6-2ea6-4126-9f5c-dd0f7c53422f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:17.951437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ed5e4-fc3c-4481-9c84-17dc3e060c95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:17.953124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ed5e4-fc3c-4481-9c84-17dc3e060c95 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:17.961793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 535/1079] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:03:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:03:17.979148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86226c6-2ea6-4126-9f5c-dd0f7c53422f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:17.980987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86226c6-2ea6-4126-9f5c-dd0f7c53422f service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:18.000183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 545/1080] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:03:20.381685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f522e3-a938-4849-94d1-5722523f832c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:20.382234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f522e3-a938-4849-94d1-5722523f832c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:20.382234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f522e3-a938-4849-94d1-5722523f832c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:20.383579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 536/1081] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:20.389463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b75521-9b22-434c-8808-e58ad8229672 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:20.389858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b75521-9b22-434c-8808-e58ad8229672 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:20.390029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b75521-9b22-434c-8808-e58ad8229672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:20.394547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:20.464714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f3b75521-9b22-434c-8808-e58ad8229672 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:20.465937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f3b75521-9b22-434c-8808-e58ad8229672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:20.575458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 546/1082] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:20.883254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f522e3-a938-4849-94d1-5722523f832c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:20.884815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-78f522e3-a938-4849-94d1-5722523f832c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:20.936388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:21.066290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f522e3-a938-4849-94d1-5722523f832c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:21.197480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:21.199807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43fc6-692f-45c7-ace7-f8fec9ee70c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:21.200629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43fc6-692f-45c7-ace7-f8fec9ee70c2 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:21.201035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43fc6-692f-45c7-ace7-f8fec9ee70c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:21.208060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43fc6-692f-45c7-ace7-f8fec9ee70c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:21.208060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-93a43fc6-692f-45c7-ace7-f8fec9ee70c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:21.308209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a43fc6-692f-45c7-ace7-f8fec9ee70c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:21.309917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a43fc6-692f-45c7-ace7-f8fec9ee70c2 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:21.332785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 537/1083] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:24.667445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46df6aa-2975-49ea-800f-94e9b54eaaba None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:24.668640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46df6aa-2975-49ea-800f-94e9b54eaaba None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:24.669164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46df6aa-2975-49ea-800f-94e9b54eaaba None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:24.670573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 547/1084] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:03:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:24.675925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce7fa0-f705-4764-8fe2-bb1768cf0edc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:24.676359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce7fa0-f705-4764-8fe2-bb1768cf0edc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:24.677066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce7fa0-f705-4764-8fe2-bb1768cf0edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:24.744831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-92ce7fa0-f705-4764-8fe2-bb1768cf0edc None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:24.746083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-92ce7fa0-f705-4764-8fe2-bb1768cf0edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:24.824805 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:24.855309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 538/1085] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:25.150457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46df6aa-2975-49ea-800f-94e9b54eaaba None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:25.152082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e46df6aa-2975-49ea-800f-94e9b54eaaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:25.249734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:25.313638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46df6aa-2975-49ea-800f-94e9b54eaaba None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:25.440400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92d5cd-6a1f-4e4c-b79c-73bb40feb6a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:25.441011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d5cd-6a1f-4e4c-b79c-73bb40feb6a2 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:25.441288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d5cd-6a1f-4e4c-b79c-73bb40feb6a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:25.441553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d5cd-6a1f-4e4c-b79c-73bb40feb6a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:25.443336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2b92d5cd-6a1f-4e4c-b79c-73bb40feb6a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:25.492054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:25.544356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92d5cd-6a1f-4e4c-b79c-73bb40feb6a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:25.546306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92d5cd-6a1f-4e4c-b79c-73bb40feb6a2 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:25.570173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 548/1086] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:34.901935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=11 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:35.564590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=26 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:42.409818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801258-99cd-41a8-ae81-bf7a5496be7a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:42.410403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801258-99cd-41a8-ae81-bf7a5496be7a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:42.410896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801258-99cd-41a8-ae81-bf7a5496be7a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:42.413149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 539/1087] 10.222.0.20 () {54 vars in 918 bytes} [Wed Oct 26 11:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:03:42.418736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766e6a8-85cb-419c-89c5-1fb0063169a8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:42.419170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766e6a8-85cb-419c-89c5-1fb0063169a8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:42.419260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766e6a8-85cb-419c-89c5-1fb0063169a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:42.497505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0766e6a8-85cb-419c-89c5-1fb0063169a8 None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:42.498722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0766e6a8-85cb-419c-89c5-1fb0063169a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:42.606328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 549/1088] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Oct 26 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:42.621402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed2319-46d8-415b-9ed0-1bdd30581b70 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:42.621685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed2319-46d8-415b-9ed0-1bdd30581b70 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:42.621959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed2319-46d8-415b-9ed0-1bdd30581b70 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:42.623301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 540/1089] 10.222.0.20 () {54 vars in 918 bytes} [Wed Oct 26 11:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:42.627805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846253-5dc3-4fd6-a309-ebe80a8869f0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:42.628097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846253-5dc3-4fd6-a309-ebe80a8869f0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:42.628366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846253-5dc3-4fd6-a309-ebe80a8869f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:42.702579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:42.705343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-52846253-5dc3-4fd6-a309-ebe80a8869f0 None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:42.706550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-52846253-5dc3-4fd6-a309-ebe80a8869f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:42.823699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 550/1090] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Oct 26 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:03:45.405868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26225f1-a413-4829-9cd6-2d5cdf0add10 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:45.406429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26225f1-a413-4829-9cd6-2d5cdf0add10 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:45.406429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26225f1-a413-4829-9cd6-2d5cdf0add10 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:45.407738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 541/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:45.413151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb291-f90e-49a1-bc0e-961b6b2246ea None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:45.413576 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb291-f90e-49a1-bc0e-961b6b2246ea None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:45.413677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb291-f90e-49a1-bc0e-961b6b2246ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:45.480770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-dd4eb291-f90e-49a1-bc0e-961b6b2246ea None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:45.481962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dd4eb291-f90e-49a1-bc0e-961b6b2246ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:45.536311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:45.577806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 551/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:45.867975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26225f1-a413-4829-9cd6-2d5cdf0add10 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:45.869542 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f26225f1-a413-4829-9cd6-2d5cdf0add10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:46.013051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26225f1-a413-4829-9cd6-2d5cdf0add10 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:46.087921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:46.126925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb60ed9-990f-4d03-9b70-776664f1672c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:46.127528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb60ed9-990f-4d03-9b70-776664f1672c service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:46.127828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb60ed9-990f-4d03-9b70-776664f1672c service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:46.128104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb60ed9-990f-4d03-9b70-776664f1672c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:46.130302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3fb60ed9-990f-4d03-9b70-776664f1672c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:46.219426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb60ed9-990f-4d03-9b70-776664f1672c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:46.221099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb60ed9-990f-4d03-9b70-776664f1672c service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:46.239457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 542/1093] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:47.367365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68273b6b-8963-4228-97e1-675945bff70a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:47.367932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68273b6b-8963-4228-97e1-675945bff70a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:47.367932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68273b6b-8963-4228-97e1-675945bff70a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:47.369298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 552/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:47.374263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5adc-3ad5-4cb0-b34f-7ae975791d85 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:47.374686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5adc-3ad5-4cb0-b34f-7ae975791d85 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:47.375025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5adc-3ad5-4cb0-b34f-7ae975791d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:47.451648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f39b5adc-3ad5-4cb0-b34f-7ae975791d85 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:47.452894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f39b5adc-3ad5-4cb0-b34f-7ae975791d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:47.480712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:47.553079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 543/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:47.709632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68273b6b-8963-4228-97e1-675945bff70a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:47.711110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-68273b6b-8963-4228-97e1-675945bff70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:47.857289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68273b6b-8963-4228-97e1-675945bff70a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:47.900355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:47.972153 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4320a3-db68-4d60-820f-29aaba3ec9ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:47.973508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4320a3-db68-4d60-820f-29aaba3ec9ce service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:47.973788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4320a3-db68-4d60-820f-29aaba3ec9ce service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:47.974055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4320a3-db68-4d60-820f-29aaba3ec9ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:47.975648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ea4320a3-db68-4d60-820f-29aaba3ec9ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:48.068277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4320a3-db68-4d60-820f-29aaba3ec9ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:48.069844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4320a3-db68-4d60-820f-29aaba3ec9ce service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:48.089666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 553/1096] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:49.227475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c59b-802d-42de-8c61-aea1bd6da9d5 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:49.228101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c59b-802d-42de-8c61-aea1bd6da9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:49.228101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c59b-802d-42de-8c61-aea1bd6da9d5 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:49.229383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 544/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:49.234136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e37bad-52ee-4b36-8129-83f4fa1ed007 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:49.234565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e37bad-52ee-4b36-8129-83f4fa1ed007 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:49.234681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e37bad-52ee-4b36-8129-83f4fa1ed007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:49.289980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:49.311599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-94e37bad-52ee-4b36-8129-83f4fa1ed007 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:49.316745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-94e37bad-52ee-4b36-8129-83f4fa1ed007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:49.408970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 554/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:49.561553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c59b-802d-42de-8c61-aea1bd6da9d5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:49.562997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bed8c59b-802d-42de-8c61-aea1bd6da9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:49.709453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c59b-802d-42de-8c61-aea1bd6da9d5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:49.721352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:49.822028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea987f-9bb3-404d-8424-49db481366cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:49.822555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea987f-9bb3-404d-8424-49db481366cf service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:49.822832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea987f-9bb3-404d-8424-49db481366cf service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:49.823102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea987f-9bb3-404d-8424-49db481366cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:49.824739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-33ea987f-9bb3-404d-8424-49db481366cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:49.912734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea987f-9bb3-404d-8424-49db481366cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:49.914318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea987f-9bb3-404d-8424-49db481366cf service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:49.931541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 545/1099] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:51.023460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe07d11-b8b4-4eba-b5d6-19448ccc09cc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:51.024076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe07d11-b8b4-4eba-b5d6-19448ccc09cc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:51.024076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe07d11-b8b4-4eba-b5d6-19448ccc09cc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:51.025371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 555/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:51.030375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad46dc6-b4bd-4872-810f-d84930b304c2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:51.030757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad46dc6-b4bd-4872-810f-d84930b304c2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:51.030960 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad46dc6-b4bd-4872-810f-d84930b304c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:51.051699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:51.108640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6ad46dc6-b4bd-4872-810f-d84930b304c2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:51.109862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6ad46dc6-b4bd-4872-810f-d84930b304c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:51.205280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 546/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:51.355237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe07d11-b8b4-4eba-b5d6-19448ccc09cc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:51.356794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6fe07d11-b8b4-4eba-b5d6-19448ccc09cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:51.466415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:51.501249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe07d11-b8b4-4eba-b5d6-19448ccc09cc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:51.613660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475ac90-9a1f-47f4-8bf5-913e57f9f151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:51.614174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ac90-9a1f-47f4-8bf5-913e57f9f151 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:51.614450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ac90-9a1f-47f4-8bf5-913e57f9f151 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:51.614720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ac90-9a1f-47f4-8bf5-913e57f9f151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:51.616391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6475ac90-9a1f-47f4-8bf5-913e57f9f151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:51.693824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:51.707171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475ac90-9a1f-47f4-8bf5-913e57f9f151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:51.709015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475ac90-9a1f-47f4-8bf5-913e57f9f151 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:51.727174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 556/1102] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:52.825808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747d3d5-7593-47a2-ba31-cc1cb5afb968 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:52.826264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747d3d5-7593-47a2-ba31-cc1cb5afb968 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:52.826353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747d3d5-7593-47a2-ba31-cc1cb5afb968 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:52.827683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 547/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:52.833152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408ea04-57d0-48f3-a167-c97730313761 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:52.833626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408ea04-57d0-48f3-a167-c97730313761 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:52.833733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408ea04-57d0-48f3-a167-c97730313761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:52.912775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6408ea04-57d0-48f3-a167-c97730313761 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:52.913979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6408ea04-57d0-48f3-a167-c97730313761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:53.011265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 557/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:53.163287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747d3d5-7593-47a2-ba31-cc1cb5afb968 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:53.164836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2747d3d5-7593-47a2-ba31-cc1cb5afb968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:53.247091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:53.309495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747d3d5-7593-47a2-ba31-cc1cb5afb968 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:53.422208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753825d9-6890-4e23-bb61-33173cfaea44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:53.422732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753825d9-6890-4e23-bb61-33173cfaea44 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:53.423008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753825d9-6890-4e23-bb61-33173cfaea44 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:53.423273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753825d9-6890-4e23-bb61-33173cfaea44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:53.424942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-753825d9-6890-4e23-bb61-33173cfaea44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:53.473956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:53.516283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753825d9-6890-4e23-bb61-33173cfaea44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:53.517879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753825d9-6890-4e23-bb61-33173cfaea44 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:53.537558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 548/1105] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:54.634532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f08e-4122-4b52-86b0-b994e492d373 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:54.635116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f08e-4122-4b52-86b0-b994e492d373 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:54.635116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f08e-4122-4b52-86b0-b994e492d373 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:54.636485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 558/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:54.641685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221d484-91fe-4224-be7a-041522802ca7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:54.642121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221d484-91fe-4224-be7a-041522802ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:54.642232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221d484-91fe-4224-be7a-041522802ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:54.717572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0221d484-91fe-4224-be7a-041522802ca7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:54.718784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0221d484-91fe-4224-be7a-041522802ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:54.814806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 549/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:54.962015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f08e-4122-4b52-86b0-b994e492d373 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:54.963530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-68c3f08e-4122-4b52-86b0-b994e492d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:55.012871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:55.110735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f08e-4122-4b52-86b0-b994e492d373 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:55.224038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11e389-9934-4c57-8bed-9ab40fe55671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:55.224562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11e389-9934-4c57-8bed-9ab40fe55671 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:55.224839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11e389-9934-4c57-8bed-9ab40fe55671 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:55.225110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11e389-9934-4c57-8bed-9ab40fe55671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:55.226699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9b11e389-9934-4c57-8bed-9ab40fe55671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:55.233477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:55.316332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11e389-9934-4c57-8bed-9ab40fe55671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:55.317931 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11e389-9934-4c57-8bed-9ab40fe55671 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:55.335943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 559/1108] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:56.444486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8defe-f3d8-4b20-9553-dc8fb85957cb None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:56.445064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8defe-f3d8-4b20-9553-dc8fb85957cb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:56.445064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8defe-f3d8-4b20-9553-dc8fb85957cb None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:56.446378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 550/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:56.451301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef75ef-fd16-4956-88d5-9bcd54ac6284 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:56.451745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef75ef-fd16-4956-88d5-9bcd54ac6284 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:56.451936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef75ef-fd16-4956-88d5-9bcd54ac6284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:56.527014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f2ef75ef-fd16-4956-88d5-9bcd54ac6284 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:56.528248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f2ef75ef-fd16-4956-88d5-9bcd54ac6284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:56.594968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:56.622531 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 560/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:03:56.773281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8defe-f3d8-4b20-9553-dc8fb85957cb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:56.774741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c6a8defe-f3d8-4b20-9553-dc8fb85957cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:56.791997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:56.922814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8defe-f3d8-4b20-9553-dc8fb85957cb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:57.007312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:57.033699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f59ce-b7ae-41ad-93b5-bf77d4f8a3f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:57.034233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f59ce-b7ae-41ad-93b5-bf77d4f8a3f4 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:57.035099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f59ce-b7ae-41ad-93b5-bf77d4f8a3f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:57.035374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f59ce-b7ae-41ad-93b5-bf77d4f8a3f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:57.037168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d50f59ce-b7ae-41ad-93b5-bf77d4f8a3f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:57.131607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f59ce-b7ae-41ad-93b5-bf77d4f8a3f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:57.133265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f59ce-b7ae-41ad-93b5-bf77d4f8a3f4 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:57.152265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 551/1111] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:03:58.281132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff48f-15fa-460e-a602-bf1eae8c81b9 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:58.281709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff48f-15fa-460e-a602-bf1eae8c81b9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:58.281709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff48f-15fa-460e-a602-bf1eae8c81b9 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:58.283095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 561/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:58.288821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bdb28-fa16-4a15-a0e1-b4ad18abf064 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:58.289245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bdb28-fa16-4a15-a0e1-b4ad18abf064 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:58.289351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bdb28-fa16-4a15-a0e1-b4ad18abf064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:58.365383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-705bdb28-fa16-4a15-a0e1-b4ad18abf064 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:58.366580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-705bdb28-fa16-4a15-a0e1-b4ad18abf064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:58.407733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:58.463290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 552/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:03:58.616403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff48f-15fa-460e-a602-bf1eae8c81b9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:58.617948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-57cff48f-15fa-460e-a602-bf1eae8c81b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:58.762816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff48f-15fa-460e-a602-bf1eae8c81b9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:58.816314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:58.873414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20d2e4-97db-49d5-8901-6df70804ed84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:58.874013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20d2e4-97db-49d5-8901-6df70804ed84 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:58.874289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20d2e4-97db-49d5-8901-6df70804ed84 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:58.874555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20d2e4-97db-49d5-8901-6df70804ed84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:58.876188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-be20d2e4-97db-49d5-8901-6df70804ed84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:58.966605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20d2e4-97db-49d5-8901-6df70804ed84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:58.968216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20d2e4-97db-49d5-8901-6df70804ed84 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:58.985806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 562/1114] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:00.092541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3969-0379-4834-9e56-14ba71d8ad15 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:00.093103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3969-0379-4834-9e56-14ba71d8ad15 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:00.093103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3969-0379-4834-9e56-14ba71d8ad15 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:00.094413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 553/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:00.099479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4566e8-6673-4988-a779-9f433ea3a734 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:00.099886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4566e8-6673-4988-a779-9f433ea3a734 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:00.100055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4566e8-6673-4988-a779-9f433ea3a734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:00.175880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4d4566e8-6673-4988-a779-9f433ea3a734 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:00.177091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4d4566e8-6673-4988-a779-9f433ea3a734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:00.179065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:00.275238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 563/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:00.423346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d3969-0379-4834-9e56-14ba71d8ad15 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:00.425441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-579d3969-0379-4834-9e56-14ba71d8ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:00.568706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d3969-0379-4834-9e56-14ba71d8ad15 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:00.596577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:00.682200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956db656-6a1b-482e-9239-25d324304ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:00.682726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db656-6a1b-482e-9239-25d324304ec5 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:00.683005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db656-6a1b-482e-9239-25d324304ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:00.683299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db656-6a1b-482e-9239-25d324304ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:00.684950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-956db656-6a1b-482e-9239-25d324304ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:00.773623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956db656-6a1b-482e-9239-25d324304ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:00.775230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956db656-6a1b-482e-9239-25d324304ec5 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:00.793399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 554/1117] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:02.857302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3e0b4-cfe5-49ed-854a-0bf4a9a485ed None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:02.857885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3e0b4-cfe5-49ed-854a-0bf4a9a485ed None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:02.857885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3e0b4-cfe5-49ed-854a-0bf4a9a485ed None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:02.859193 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 564/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:02.865378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535e80a-9000-4618-9834-533d15cb28b9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:02.865856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535e80a-9000-4618-9834-533d15cb28b9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:02.865959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535e80a-9000-4618-9834-533d15cb28b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:02.909229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:02.934254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c535e80a-9000-4618-9834-533d15cb28b9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:02.935455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c535e80a-9000-4618-9834-533d15cb28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:03.030626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 555/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:03.318699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3e0b4-cfe5-49ed-854a-0bf4a9a485ed None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:03.322478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4fb3e0b4-cfe5-49ed-854a-0bf4a9a485ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:03.425957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3e0b4-cfe5-49ed-854a-0bf4a9a485ed None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:03.459392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:03.499732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249419f7-622a-4d43-976a-cf27ba0027dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:03.500391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249419f7-622a-4d43-976a-cf27ba0027dd service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:03.500667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249419f7-622a-4d43-976a-cf27ba0027dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:03.500935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249419f7-622a-4d43-976a-cf27ba0027dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:03.502515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-249419f7-622a-4d43-976a-cf27ba0027dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:03.610294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249419f7-622a-4d43-976a-cf27ba0027dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:03.611716 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249419f7-622a-4d43-976a-cf27ba0027dd service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:03.630475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 565/1120] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:05.942425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd7459-90c8-48e4-8f81-84dfdac8e49c None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:05.942976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd7459-90c8-48e4-8f81-84dfdac8e49c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:05.942976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd7459-90c8-48e4-8f81-84dfdac8e49c None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:05.944382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 556/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:05.949591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c5795-96e6-48cc-a717-475c2f56a864 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:05.949999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c5795-96e6-48cc-a717-475c2f56a864 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:05.950131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c5795-96e6-48cc-a717-475c2f56a864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:06.015186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3c1c5795-96e6-48cc-a717-475c2f56a864 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:06.016339 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3c1c5795-96e6-48cc-a717-475c2f56a864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:06.029405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:06.114022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 566/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:06.399243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd7459-90c8-48e4-8f81-84dfdac8e49c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:06.400804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-77cd7459-90c8-48e4-8f81-84dfdac8e49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:06.505833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd7459-90c8-48e4-8f81-84dfdac8e49c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:06.519292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:06.587509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d293a3-5bc1-4879-93bf-8e39813b2e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:06.588136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d293a3-5bc1-4879-93bf-8e39813b2e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:06.588410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d293a3-5bc1-4879-93bf-8e39813b2e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:06.588680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d293a3-5bc1-4879-93bf-8e39813b2e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:06.590263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-27d293a3-5bc1-4879-93bf-8e39813b2e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:06.701922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d293a3-5bc1-4879-93bf-8e39813b2e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:06.703356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d293a3-5bc1-4879-93bf-8e39813b2e8f service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:06.722561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 557/1123] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:09.267810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe9a5-d1e1-40d1-93c8-4cb6327c7a39 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:09.268495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe9a5-d1e1-40d1-93c8-4cb6327c7a39 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:09.268495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe9a5-d1e1-40d1-93c8-4cb6327c7a39 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:09.270284 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 567/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:09.275679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f1e9b-4c45-4f79-80b5-647db21ac35e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:09.276141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f1e9b-4c45-4f79-80b5-647db21ac35e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:09.276251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f1e9b-4c45-4f79-80b5-647db21ac35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:09.324857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:09.341068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7e4f1e9b-4c45-4f79-80b5-647db21ac35e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:09.342275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7e4f1e9b-4c45-4f79-80b5-647db21ac35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:09.437289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 558/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:09.723253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe9a5-d1e1-40d1-93c8-4cb6327c7a39 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:09.724846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-520fe9a5-d1e1-40d1-93c8-4cb6327c7a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:09.826601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe9a5-d1e1-40d1-93c8-4cb6327c7a39 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:09.866127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:09.899774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b669ac-36fe-418d-a6c7-49cee4250ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:09.900394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b669ac-36fe-418d-a6c7-49cee4250ade service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:09.900668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b669ac-36fe-418d-a6c7-49cee4250ade service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:09.900933 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b669ac-36fe-418d-a6c7-49cee4250ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:09.902497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-84b669ac-36fe-418d-a6c7-49cee4250ade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:10.008253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b669ac-36fe-418d-a6c7-49cee4250ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:10.009611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b669ac-36fe-418d-a6c7-49cee4250ade service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:10.028486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 568/1126] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:12.310937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28fa0-dd2f-4469-ba54-067276e9dc81 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:12.311472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28fa0-dd2f-4469-ba54-067276e9dc81 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:12.311472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28fa0-dd2f-4469-ba54-067276e9dc81 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:12.312830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 559/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:12.318719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c6304-d303-4917-b375-37068073e31a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:12.323427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c6304-d303-4917-b375-37068073e31a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:12.323427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c6304-d303-4917-b375-37068073e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:12.385619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-836c6304-d303-4917-b375-37068073e31a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:12.387092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-836c6304-d303-4917-b375-37068073e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:12.408149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:12.492457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 569/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:12.795493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28fa0-dd2f-4469-ba54-067276e9dc81 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:12.797052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bad28fa0-dd2f-4469-ba54-067276e9dc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:12.903162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28fa0-dd2f-4469-ba54-067276e9dc81 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:12.924956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:12.978192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0bc1e-fe5d-443c-9d09-b2a579cc1662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:12.978811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0bc1e-fe5d-443c-9d09-b2a579cc1662 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:12.979085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0bc1e-fe5d-443c-9d09-b2a579cc1662 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:12.979352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0bc1e-fe5d-443c-9d09-b2a579cc1662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:12.980960 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2aa0bc1e-fe5d-443c-9d09-b2a579cc1662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:13.088884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0bc1e-fe5d-443c-9d09-b2a579cc1662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:13.090263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0bc1e-fe5d-443c-9d09-b2a579cc1662 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:13.108529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 560/1129] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:15.316422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0dd8-0290-46b1-a284-be8e8e620eaa None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:15.317092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0dd8-0290-46b1-a284-be8e8e620eaa None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:15.317092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0dd8-0290-46b1-a284-be8e8e620eaa None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:15.318297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 570/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:15.323552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85deabc-ad4b-4441-aa16-a1b085a6c5af None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:15.327476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85deabc-ad4b-4441-aa16-a1b085a6c5af None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:15.327476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85deabc-ad4b-4441-aa16-a1b085a6c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:15.388281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:15.390804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b85deabc-ad4b-4441-aa16-a1b085a6c5af None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:15.392024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b85deabc-ad4b-4441-aa16-a1b085a6c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:15.489829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 561/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:15.779533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0dd8-0290-46b1-a284-be8e8e620eaa None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:15.781102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bf8a0dd8-0290-46b1-a284-be8e8e620eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:15.885635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0dd8-0290-46b1-a284-be8e8e620eaa None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:15.934339 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:15.958609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783333e-f46b-43fc-9bf8-6dbe587a4dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:15.959200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783333e-f46b-43fc-9bf8-6dbe587a4dad service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:15.959478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783333e-f46b-43fc-9bf8-6dbe587a4dad service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:15.959745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783333e-f46b-43fc-9bf8-6dbe587a4dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:15.961393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f783333e-f46b-43fc-9bf8-6dbe587a4dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:16.071509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783333e-f46b-43fc-9bf8-6dbe587a4dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:16.072973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783333e-f46b-43fc-9bf8-6dbe587a4dad service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:16.091613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 571/1132] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:18.326143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42cd91-7955-4370-9bb6-58e9df2bd555 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:18.326760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42cd91-7955-4370-9bb6-58e9df2bd555 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:18.326760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42cd91-7955-4370-9bb6-58e9df2bd555 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:18.327986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 562/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:18.334884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70717194-7cb4-43d6-bd58-361685041061 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:18.335346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70717194-7cb4-43d6-bd58-361685041061 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:18.335447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70717194-7cb4-43d6-bd58-361685041061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:18.402561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-70717194-7cb4-43d6-bd58-361685041061 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:18.403750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-70717194-7cb4-43d6-bd58-361685041061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:18.433834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:18.500960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 572/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:18.789783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42cd91-7955-4370-9bb6-58e9df2bd555 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:18.792346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5e42cd91-7955-4370-9bb6-58e9df2bd555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:18.894739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42cd91-7955-4370-9bb6-58e9df2bd555 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:18.923553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:18.969256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6306f1-63ff-4170-9fd9-d7aa7c8f092e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:18.969860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6306f1-63ff-4170-9fd9-d7aa7c8f092e service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:18.970135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6306f1-63ff-4170-9fd9-d7aa7c8f092e service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:18.970431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6306f1-63ff-4170-9fd9-d7aa7c8f092e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:18.972100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3b6306f1-63ff-4170-9fd9-d7aa7c8f092e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:19.079316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6306f1-63ff-4170-9fd9-d7aa7c8f092e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:19.081475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6306f1-63ff-4170-9fd9-d7aa7c8f092e service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:19.099721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 563/1135] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:04:21.409956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c56e-d94d-43da-a7c8-51292e14fbd4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:21.410509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c56e-d94d-43da-a7c8-51292e14fbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:21.410509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c56e-d94d-43da-a7c8-51292e14fbd4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:21.411927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 573/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:04:21.417362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeea66-cb1c-4b19-bc4b-8d503f3d28b6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:21.417768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeea66-cb1c-4b19-bc4b-8d503f3d28b6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:21.417911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeea66-cb1c-4b19-bc4b-8d503f3d28b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:21.484639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-dfdeea66-cb1c-4b19-bc4b-8d503f3d28b6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:21.485846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-dfdeea66-cb1c-4b19-bc4b-8d503f3d28b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:21.494751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:21.584471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 564/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:21.875811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7c56e-d94d-43da-a7c8-51292e14fbd4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:21.877498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5be7c56e-d94d-43da-a7c8-51292e14fbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:21.979703 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7c56e-d94d-43da-a7c8-51292e14fbd4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:22.035961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:22.054976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47ceb-9e86-406f-b2b8-78c58ded922d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:22.055580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47ceb-9e86-406f-b2b8-78c58ded922d service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:22.055892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47ceb-9e86-406f-b2b8-78c58ded922d service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:22.056161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47ceb-9e86-406f-b2b8-78c58ded922d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:22.057730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5ea47ceb-9e86-406f-b2b8-78c58ded922d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:22.164434 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47ceb-9e86-406f-b2b8-78c58ded922d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:22.165861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47ceb-9e86-406f-b2b8-78c58ded922d service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:22.184447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 574/1138] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:24.421864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f6958-4755-41c4-bdf2-d25377e12298 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:24.422403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f6958-4755-41c4-bdf2-d25377e12298 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:24.422403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f6958-4755-41c4-bdf2-d25377e12298 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:24.423741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 565/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:24.429976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82aeb9-b727-4e6b-be1e-db50ca6378f3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:24.430369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82aeb9-b727-4e6b-be1e-db50ca6378f3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:24.430513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82aeb9-b727-4e6b-be1e-db50ca6378f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:24.498603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6b82aeb9-b727-4e6b-be1e-db50ca6378f3 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:24.499809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6b82aeb9-b727-4e6b-be1e-db50ca6378f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:24.537838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:24.595138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 575/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:24.885509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f6958-4755-41c4-bdf2-d25377e12298 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:24.887289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5f1f6958-4755-41c4-bdf2-d25377e12298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:24.990781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f6958-4755-41c4-bdf2-d25377e12298 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:25.028216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:25.067003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de32fe1-14b7-4919-b1ce-e0e74a14e47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:25.067729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de32fe1-14b7-4919-b1ce-e0e74a14e47b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:25.068176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de32fe1-14b7-4919-b1ce-e0e74a14e47b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:25.068564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de32fe1-14b7-4919-b1ce-e0e74a14e47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:25.070290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1de32fe1-14b7-4919-b1ce-e0e74a14e47b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:25.176974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de32fe1-14b7-4919-b1ce-e0e74a14e47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:25.178399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de32fe1-14b7-4919-b1ce-e0e74a14e47b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:25.197081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 566/1141] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:27.375568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b821a8-3768-4985-8cc7-bdd2b785a2f2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:27.376166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b821a8-3768-4985-8cc7-bdd2b785a2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:27.376166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b821a8-3768-4985-8cc7-bdd2b785a2f2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:27.378221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 576/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:27.383475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c049a8-a0aa-4ee4-afc2-8315b3c658ce None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:27.383936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c049a8-a0aa-4ee4-afc2-8315b3c658ce None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:27.384125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c049a8-a0aa-4ee4-afc2-8315b3c658ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:27.451059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-27c049a8-a0aa-4ee4-afc2-8315b3c658ce None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:27.452379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-27c049a8-a0aa-4ee4-afc2-8315b3c658ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:27.470597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:27.551750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 567/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:27.835065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b821a8-3768-4985-8cc7-bdd2b785a2f2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:27.836702 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-55b821a8-3768-4985-8cc7-bdd2b785a2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:27.940798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b821a8-3768-4985-8cc7-bdd2b785a2f2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:28.007994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:28.014840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385f394-1067-44aa-bd5d-dbc2a58b0bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:28.015441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385f394-1067-44aa-bd5d-dbc2a58b0bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:28.015719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385f394-1067-44aa-bd5d-dbc2a58b0bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:28.032103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385f394-1067-44aa-bd5d-dbc2a58b0bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:28.032103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e385f394-1067-44aa-bd5d-dbc2a58b0bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:28.127190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385f394-1067-44aa-bd5d-dbc2a58b0bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:28.129283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385f394-1067-44aa-bd5d-dbc2a58b0bb8 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:28.148618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 577/1144] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:30.323597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760a05-976b-415b-9560-8cf3fedc25c3 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:30.323597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760a05-976b-415b-9560-8cf3fedc25c3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:30.323597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760a05-976b-415b-9560-8cf3fedc25c3 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:30.323597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 568/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:30.327000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ecb45-9cdc-4021-8d1a-e036bd7ac018 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:30.327438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ecb45-9cdc-4021-8d1a-e036bd7ac018 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:30.327536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ecb45-9cdc-4021-8d1a-e036bd7ac018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:30.392146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ab7ecb45-9cdc-4021-8d1a-e036bd7ac018 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:30.393471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ab7ecb45-9cdc-4021-8d1a-e036bd7ac018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:30.440775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:30.488758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 578/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:30.771229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760a05-976b-415b-9560-8cf3fedc25c3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:30.772811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3c760a05-976b-415b-9560-8cf3fedc25c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:30.874874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760a05-976b-415b-9560-8cf3fedc25c3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:30.922700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:30.947806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89348c20-d873-47a0-9fb0-2908f4317149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:30.948412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89348c20-d873-47a0-9fb0-2908f4317149 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:30.948690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89348c20-d873-47a0-9fb0-2908f4317149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:30.948964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89348c20-d873-47a0-9fb0-2908f4317149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:30.950594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-89348c20-d873-47a0-9fb0-2908f4317149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:31.061037 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89348c20-d873-47a0-9fb0-2908f4317149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:31.062467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89348c20-d873-47a0-9fb0-2908f4317149 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:31.080262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 569/1147] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:04:40.486778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=22 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:41.077927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=66 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:51.135317 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3e533-9c72-472f-8a99-01931bb14d7d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:51.136170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3e533-9c72-472f-8a99-01931bb14d7d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:51.136270 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3e533-9c72-472f-8a99-01931bb14d7d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:51.137589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 579/1148] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:51.143537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640e79-3ede-48ca-b8b4-d38ef5f55405 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:51.143964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640e79-3ede-48ca-b8b4-d38ef5f55405 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:51.144113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640e79-3ede-48ca-b8b4-d38ef5f55405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:51.208381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-13640e79-3ede-48ca-b8b4-d38ef5f55405 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:51.209581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-13640e79-3ede-48ca-b8b4-d38ef5f55405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:51.304600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 570/1149] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:51.604230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ee0ee-fe73-43f7-a235-63b56699c473 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:51.604520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ee0ee-fe73-43f7-a235-63b56699c473 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:51.604798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ee0ee-fe73-43f7-a235-63b56699c473 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:51.606157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 580/1150] 10.222.0.20 () {54 vars in 962 bytes} [Wed Oct 26 11:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:51.612055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed112626-8f18-434e-97d7-93f479b53765 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:51.612348 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed112626-8f18-434e-97d7-93f479b53765 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:51.612624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed112626-8f18-434e-97d7-93f479b53765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:51.689867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ed112626-8f18-434e-97d7-93f479b53765 None None] MFA Rules not processed for user `9627e6a9e86e43b694f723bd4109460e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:51.691066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ed112626-8f18-434e-97d7-93f479b53765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:51.711165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:51.768166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 571/1151] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Oct 26 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:51.775971 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ee0ee-fe73-43f7-a235-63b56699c473 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:51.777522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a39ee0ee-fe73-43f7-a235-63b56699c473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:51.881773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ee0ee-fe73-43f7-a235-63b56699c473 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:51.954984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c1802-4746-4093-ac89-95ba14b910de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9627e6a9e86e43b694f723bd4109460e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:51.955581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1802-4746-4093-ac89-95ba14b910de service glance] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:51.955882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1802-4746-4093-ac89-95ba14b910de service glance] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:51.956154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1802-4746-4093-ac89-95ba14b910de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:51.957724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-525c1802-4746-4093-ac89-95ba14b910de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:52.011380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:52.068165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c1802-4746-4093-ac89-95ba14b910de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:52.069586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c1802-4746-4093-ac89-95ba14b910de service glance] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:52.087584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 581/1152] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:04:54.334739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c6eb-53ec-4dda-9ad2-f5b665f06de1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:54.335249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c6eb-53ec-4dda-9ad2-f5b665f06de1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:54.335249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c6eb-53ec-4dda-9ad2-f5b665f06de1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:54.336597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 572/1153] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:54.341943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37751c1c-2d88-44b0-8fad-7bd5666c5d6b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:54.342368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37751c1c-2d88-44b0-8fad-7bd5666c5d6b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:54.342479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37751c1c-2d88-44b0-8fad-7bd5666c5d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:54.409679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-37751c1c-2d88-44b0-8fad-7bd5666c5d6b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:54.410859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-37751c1c-2d88-44b0-8fad-7bd5666c5d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:54.505217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 582/1154] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:54.778995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7c6eb-53ec-4dda-9ad2-f5b665f06de1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:54.780564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a8c7c6eb-53ec-4dda-9ad2-f5b665f06de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:54.906227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7c6eb-53ec-4dda-9ad2-f5b665f06de1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:54.956712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:55.002019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ea055-8ef6-47e3-8d31-66298ee0229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:55.002618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ea055-8ef6-47e3-8d31-66298ee0229c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:55.002898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ea055-8ef6-47e3-8d31-66298ee0229c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:55.003173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ea055-8ef6-47e3-8d31-66298ee0229c admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:55.003809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ea055-8ef6-47e3-8d31-66298ee0229c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:55.004982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-c76ea055-8ef6-47e3-8d31-66298ee0229c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:04:55.004982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 11:04:55.004982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 11:04:55.006108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ea055-8ef6-47e3-8d31-66298ee0229c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:55.026036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 573/1155] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 26 11:04:54 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:04:57.181333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90d5e-820f-4d0a-b1f7-0e93c210282b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:57.181917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90d5e-820f-4d0a-b1f7-0e93c210282b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:57.181917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a90d5e-820f-4d0a-b1f7-0e93c210282b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:57.183205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 583/1156] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:04:57.189221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ba25-e584-4bad-8f6e-a1175fbdcf36 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:57.189663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ba25-e584-4bad-8f6e-a1175fbdcf36 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:57.189975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ba25-e584-4bad-8f6e-a1175fbdcf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:57.257189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-01e0ba25-e584-4bad-8f6e-a1175fbdcf36 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:04:57.258394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-01e0ba25-e584-4bad-8f6e-a1175fbdcf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:57.354851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 574/1157] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:04:57.653974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a90d5e-820f-4d0a-b1f7-0e93c210282b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:04:57.655513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-00a90d5e-820f-4d0a-b1f7-0e93c210282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:57.665391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:57.801328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a90d5e-820f-4d0a-b1f7-0e93c210282b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:04:57.880951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:57.913639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aa1cf-8c7f-47b5-8ca9-f49cfa102233 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:04:57.914175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aa1cf-8c7f-47b5-8ca9-f49cfa102233 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:04:57.914453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aa1cf-8c7f-47b5-8ca9-f49cfa102233 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:04:57.914725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aa1cf-8c7f-47b5-8ca9-f49cfa102233 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:04:57.916342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fc5aa1cf-8c7f-47b5-8ca9-f49cfa102233 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:04:58.005062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aa1cf-8c7f-47b5-8ca9-f49cfa102233 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:04:58.006754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aa1cf-8c7f-47b5-8ca9-f49cfa102233 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:04:58.025363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 584/1158] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:05:00.367595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855618-a7bf-400e-b7d2-e057d0afed9e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:00.369770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855618-a7bf-400e-b7d2-e057d0afed9e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:00.369770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855618-a7bf-400e-b7d2-e057d0afed9e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:00.369770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 575/1159] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:05:00.374726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12d44-eed4-494f-8a4d-d49565629938 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:00.375176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12d44-eed4-494f-8a4d-d49565629938 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:00.375321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12d44-eed4-494f-8a4d-d49565629938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:00.441481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0ba12d44-eed4-494f-8a4d-d49565629938 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:05:00.442691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0ba12d44-eed4-494f-8a4d-d49565629938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:00.489593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:00.540069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 585/1160] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:05:00.828497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2855618-a7bf-400e-b7d2-e057d0afed9e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:05:00.830755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c2855618-a7bf-400e-b7d2-e057d0afed9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:00.854265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:00.934363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2855618-a7bf-400e-b7d2-e057d0afed9e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:05:01.010547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abd07c-ad5d-44ff-a698-70d5f8f3fc0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:05:01.011393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abd07c-ad5d-44ff-a698-70d5f8f3fc0c service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:01.011668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abd07c-ad5d-44ff-a698-70d5f8f3fc0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:01.011971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abd07c-ad5d-44ff-a698-70d5f8f3fc0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:01.013534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b4abd07c-ad5d-44ff-a698-70d5f8f3fc0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:01.089659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:01.123247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abd07c-ad5d-44ff-a698-70d5f8f3fc0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:05:01.124759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abd07c-ad5d-44ff-a698-70d5f8f3fc0c service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:05:01.144322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 576/1161] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:05:03.348018 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ad05-9be1-4d1a-87a0-b80c90c22248 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:03.348566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ad05-9be1-4d1a-87a0-b80c90c22248 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:03.348566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ad05-9be1-4d1a-87a0-b80c90c22248 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:03.349828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 586/1162] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:05:03.355022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38a34-4391-4e5c-a523-3d142c52dfc5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:03.355437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38a34-4391-4e5c-a523-3d142c52dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:03.355573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38a34-4391-4e5c-a523-3d142c52dfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:03.420810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f4d38a34-4391-4e5c-a523-3d142c52dfc5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:05:03.422003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f4d38a34-4391-4e5c-a523-3d142c52dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:03.519330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 577/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:05:03.818850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ad05-9be1-4d1a-87a0-b80c90c22248 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:05:03.820408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3c37ad05-9be1-4d1a-87a0-b80c90c22248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:03.922799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37ad05-9be1-4d1a-87a0-b80c90c22248 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:05:03.994215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:03.995683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dcf22-d64d-43be-a753-832d7238dd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:05:03.996385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dcf22-d64d-43be-a753-832d7238dd7b service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:03.996663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dcf22-d64d-43be-a753-832d7238dd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:03.996933 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dcf22-d64d-43be-a753-832d7238dd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:03.998522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ea0dcf22-d64d-43be-a753-832d7238dd7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:04.108472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dcf22-d64d-43be-a753-832d7238dd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:05:04.109925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dcf22-d64d-43be-a753-832d7238dd7b service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:05:04.128938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 587/1164] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:05:06.421800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfd670-b126-4db2-8b62-d08615380854 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:06.422898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfd670-b126-4db2-8b62-d08615380854 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:06.422898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfd670-b126-4db2-8b62-d08615380854 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:06.424269 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 578/1165] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:05:06.429940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a5c7-1a3b-483a-9a41-779eb9ce6b69 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:06.430362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a5c7-1a3b-483a-9a41-779eb9ce6b69 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:06.430535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a5c7-1a3b-483a-9a41-779eb9ce6b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:06.501885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4b80a5c7-1a3b-483a-9a41-779eb9ce6b69 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:05:06.503118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4b80a5c7-1a3b-483a-9a41-779eb9ce6b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:06.558471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:06.608379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 588/1166] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:05:06.910654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfd670-b126-4db2-8b62-d08615380854 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:05:06.912239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-63bfd670-b126-4db2-8b62-d08615380854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:06.951263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:07.021524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfd670-b126-4db2-8b62-d08615380854 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:05:07.110351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd257f0-0e4e-463d-bc74-f95e765a2b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:05:07.110928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd257f0-0e4e-463d-bc74-f95e765a2b9c service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:07.111212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd257f0-0e4e-463d-bc74-f95e765a2b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:07.111487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd257f0-0e4e-463d-bc74-f95e765a2b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:07.113179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0cd257f0-0e4e-463d-bc74-f95e765a2b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:07.205821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:07.236315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd257f0-0e4e-463d-bc74-f95e765a2b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:05:07.238172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd257f0-0e4e-463d-bc74-f95e765a2b9c service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:05:07.257761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 579/1167] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:05:09.832484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300cc0-e530-4337-8231-c616eeef1b28 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:09.833040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300cc0-e530-4337-8231-c616eeef1b28 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:09.833040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300cc0-e530-4337-8231-c616eeef1b28 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:09.834396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 589/1168] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 26 11:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:05:09.839765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969a7bb-51b8-42f3-a276-6c0d9100b3a6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:09.840228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969a7bb-51b8-42f3-a276-6c0d9100b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:09.840359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969a7bb-51b8-42f3-a276-6c0d9100b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:09.907018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1969a7bb-51b8-42f3-a276-6c0d9100b3a6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:05:09.908429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1969a7bb-51b8-42f3-a276-6c0d9100b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:10.002917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 580/1169] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 26 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:05:10.291911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300cc0-e530-4337-8231-c616eeef1b28 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:05:10.293387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-70300cc0-e530-4337-8231-c616eeef1b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:10.395937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300cc0-e530-4337-8231-c616eeef1b28 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:05:10.468873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d58f7e-4f1a-4595-aa30-e04fc8754c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:05:10.469998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d58f7e-4f1a-4595-aa30-e04fc8754c44 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:05:10.470277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d58f7e-4f1a-4595-aa30-e04fc8754c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:05:10.470544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d58f7e-4f1a-4595-aa30-e04fc8754c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:05:10.472167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e0d58f7e-4f1a-4595-aa30-e04fc8754c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:05:10.480743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:10.581966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d58f7e-4f1a-4595-aa30-e04fc8754c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:05:10.583312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d58f7e-4f1a-4595-aa30-e04fc8754c44 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:05:10.601117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 590/1170] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:05:20.001443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=82 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:05:20.598855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=50 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:06:58.780435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942348e-1ea7-48d7-950d-9c4cb67718bd None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:58.781030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942348e-1ea7-48d7-950d-9c4cb67718bd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:58.781030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942348e-1ea7-48d7-950d-9c4cb67718bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:58.862061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7942348e-1ea7-48d7-950d-9c4cb67718bd None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:06:58.863257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7942348e-1ea7-48d7-950d-9c4cb67718bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:06:58.965484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 581/1171] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:06:58.973422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61094162-dbfb-4a1b-a8a7-42deae2b1bf0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:58.973899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61094162-dbfb-4a1b-a8a7-42deae2b1bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:58.973993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61094162-dbfb-4a1b-a8a7-42deae2b1bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:58.995079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-61094162-dbfb-4a1b-a8a7-42deae2b1bf0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:06:58.996330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-61094162-dbfb-4a1b-a8a7-42deae2b1bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:06:59.005577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 591/1172] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:06:59.010205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942348e-1ea7-48d7-950d-9c4cb67718bd None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.020212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942348e-1ea7-48d7-950d-9c4cb67718bd None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.024754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.025346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.025624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.025896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.026537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:06:59.026839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.027902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:06:59.027902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 11:06:59.027902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 11:06:59.028816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:06:59.040539 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 582/1173] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:06:59.050180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61094162-dbfb-4a1b-a8a7-42deae2b1bf0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.059128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61094162-dbfb-4a1b-a8a7-42deae2b1bf0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.064302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.064893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.065166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.065433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.066066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.067143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:06:59.067143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 11:06:59.067143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 11:06:59.068348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:06:59.171322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 592/1174] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:06:59 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:06:59.176406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.186232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9d039-102b-48e9-ac86-a0ad41c47412 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.237445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:06:59.244060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.244654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.244931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.245201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.245830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.246831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:06:59.246831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 11:06:59.246831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 11:06:59.248051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:06:59.373748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 583/1175] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:06:59 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:06:59.377923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.388182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76705991-95e7-4069-895f-1aa6503f28ca admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.392955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.393543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.393816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.394083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.394709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.396334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:06:59.409416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 593/1176] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:06:59.413269 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.424081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4788cf-b089-4259-b4b5-7a65623994f7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.428669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b19b8-6f99-443e-811d-e29d03a65f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.429284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b19b8-6f99-443e-811d-e29d03a65f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.429535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b19b8-6f99-443e-811d-e29d03a65f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.429805 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b19b8-6f99-443e-811d-e29d03a65f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.430432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b19b8-6f99-443e-811d-e29d03a65f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.432058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b19b8-6f99-443e-811d-e29d03a65f53 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:06:59.445301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 584/1177] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:06:59.449269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.458901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a85a3-2057-4bd8-82a7-7e8e3ec9e11d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.463318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefaa3d-08c3-4b5a-a7b1-0f4d7e0d7c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.463982 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefaa3d-08c3-4b5a-a7b1-0f4d7e0d7c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.464258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefaa3d-08c3-4b5a-a7b1-0f4d7e0d7c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.464529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefaa3d-08c3-4b5a-a7b1-0f4d7e0d7c66 admin admin] PATH_INFO: `/v3/projects/1e8fc239e5db4c4aada4b01c7a594db6/users/afaf5a47b2da449fae70b0fd1b20e4cc/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.479288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefaa3d-08c3-4b5a-a7b1-0f4d7e0d7c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8fc239e5db4c4aada4b01c7a594db6, user_id=afaf5a47b2da449fae70b0fd1b20e4cc, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.481927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefaa3d-08c3-4b5a-a7b1-0f4d7e0d7c66 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:06:59.551647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 594/1178] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:06:59 2022] PUT /identity/v3/projects/1e8fc239e5db4c4aada4b01c7a594db6/users/afaf5a47b2da449fae70b0fd1b20e4cc/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:06:59.563167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7215e89-5697-43e1-8b71-e0469b022902 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.563458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7215e89-5697-43e1-8b71-e0469b022902 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.563732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7215e89-5697-43e1-8b71-e0469b022902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.591625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f7215e89-5697-43e1-8b71-e0469b022902 None None] MFA Rules not processed for user `afaf5a47b2da449fae70b0fd1b20e4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:06:59.592865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f7215e89-5697-43e1-8b71-e0469b022902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:06:59.659317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 585/1179] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:06:59.672951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2be0f2-d0bb-468f-8704-5305feac7d50 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.673235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2be0f2-d0bb-468f-8704-5305feac7d50 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.673508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2be0f2-d0bb-468f-8704-5305feac7d50 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.674832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 595/1180] 10.222.0.20 () {54 vars in 963 bytes} [Wed Oct 26 11:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:06:59.679847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580be85d-b32d-4a3e-9245-297e84b3544d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.685132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580be85d-b32d-4a3e-9245-297e84b3544d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.685132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580be85d-b32d-4a3e-9245-297e84b3544d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.740331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-580be85d-b32d-4a3e-9245-297e84b3544d None None] MFA Rules not processed for user `37b474a596cf42efb5e0e5a2a179735e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:06:59.741551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-580be85d-b32d-4a3e-9245-297e84b3544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:06:59.816683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 586/1181] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Oct 26 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:06:59.824425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2be0f2-d0bb-468f-8704-5305feac7d50 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.836051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2be0f2-d0bb-468f-8704-5305feac7d50 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.839528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.840155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.840435 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.840706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.849743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.851183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:06:59.860082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 596/1182] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:06:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:06:59.906229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc663c-ed1e-4725-9363-f079bf7fb3b7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.914499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc663c-ed1e-4725-9363-f079bf7fb3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.914499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc663c-ed1e-4725-9363-f079bf7fb3b7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.914499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 587/1183] 10.222.0.20 () {52 vars in 830 bytes} [Wed Oct 26 11:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:06:59.914839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:06:59.974488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7134ffb-f58e-4c81-80e6-563c302f0a24 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:06:59.979068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:06:59.979690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:06:59.979997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:06:59.980270 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] PATH_INFO: `/v3/users/afaf5a47b2da449fae70b0fd1b20e4cc` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:06:59.981379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afaf5a47b2da449fae70b0fd1b20e4cc)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:06:59.982386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:06:59.982386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 11:06:59.982386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 11:06:59.983636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:07:00.173901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afaf5a47b2da449fae70b0fd1b20e4cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:00.174570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afaf5a47b2da449fae70b0fd1b20e4cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:00.272962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afaf5a47b2da449fae70b0fd1b20e4cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:00.273598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afaf5a47b2da449fae70b0fd1b20e4cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:00.287808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afaf5a47b2da449fae70b0fd1b20e4cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:00.310229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:07:00.318795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 597/1184] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:06:59 2022] DELETE /identity/v3/users/afaf5a47b2da449fae70b0fd1b20e4cc => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:07:00.323155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:07:00.324314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:07:00.324314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:07:00.342565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:07:00.343769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:07:00.352930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:07:00.402466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 588/1185] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:07:00.412421 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:07:00.413829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:07:00.493120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6b2da-7ab1-4c19-9c00-f4c3f04533d0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:07:00.496693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4dc31-2a5c-4bad-afd3-0e1c0cdc94d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:07:00.497304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4dc31-2a5c-4bad-afd3-0e1c0cdc94d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:07:00.497580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4dc31-2a5c-4bad-afd3-0e1c0cdc94d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:07:00.497852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4dc31-2a5c-4bad-afd3-0e1c0cdc94d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:07:00.507266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4dc31-2a5c-4bad-afd3-0e1c0cdc94d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:07:00.508972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4dc31-2a5c-4bad-afd3-0e1c0cdc94d8 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:07:00.518423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 598/1186] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:07:01.189958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:07:01.191425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:07:01.203837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5923ec5-8888-4f0d-9b2f-d47b1bfef9d1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:07:01.208724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:07:01.209370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:07:01.209710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:07:01.209890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] PATH_INFO: `/v3/projects/1e8fc239e5db4c4aada4b01c7a594db6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:07:01.210993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8fc239e5db4c4aada4b01c7a594db6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:07:01.212022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:07:01.212022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 11:07:01.212022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 11:07:01.215916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:07:01.353521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8fc239e5db4c4aada4b01c7a594db6', 'request_id': 'req-a942febb-fb83-4630-b525-16360b775f42'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:01.359477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8fc239e5db4c4aada4b01c7a594db6', 'request_id': 'req-a942febb-fb83-4630-b525-16360b775f42'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:01.372802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8fc239e5db4c4aada4b01c7a594db6', 'request_id': 'req-a942febb-fb83-4630-b525-16360b775f42'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:01.466295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] The token cache is being invalidate because project 1e8fc239e5db4c4aada4b01c7a594db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:07:01.466585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:01.467203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a942febb-fb83-4630-b525-16360b775f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:07:01.469249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 589/1187] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:07:01 2022] DELETE /identity/v3/projects/1e8fc239e5db4c4aada4b01c7a594db6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:07:10.514043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:07:11.463183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:08:26.004256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72315855-9f82-41d5-89c9-3c8bdf63d4b8 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:26.004839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72315855-9f82-41d5-89c9-3c8bdf63d4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:26.004839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72315855-9f82-41d5-89c9-3c8bdf63d4b8 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:26.006482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 599/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:26.011190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b688d7-52f0-4c34-be02-895063c74091 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:26.011644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b688d7-52f0-4c34-be02-895063c74091 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:26.011750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b688d7-52f0-4c34-be02-895063c74091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:26.033881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-72b688d7-52f0-4c34-be02-895063c74091 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:26.035146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-72b688d7-52f0-4c34-be02-895063c74091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:26.095208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 590/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:26.251282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72315855-9f82-41d5-89c9-3c8bdf63d4b8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:26.253142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-72315855-9f82-41d5-89c9-3c8bdf63d4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:26.355250 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72315855-9f82-41d5-89c9-3c8bdf63d4b8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:26.361120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f20cad-2214-4755-b21c-10eed79ef5ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:26.361709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f20cad-2214-4755-b21c-10eed79ef5ab service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:26.361983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f20cad-2214-4755-b21c-10eed79ef5ab service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:26.362253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f20cad-2214-4755-b21c-10eed79ef5ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:26.370179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f20cad-2214-4755-b21c-10eed79ef5ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:26.371821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f20cad-2214-4755-b21c-10eed79ef5ab service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:26.381650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 600/1190] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:27.459427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069df86e-54c4-498d-9eae-e51a31bfdcee None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:27.460028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069df86e-54c4-498d-9eae-e51a31bfdcee None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:27.460028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069df86e-54c4-498d-9eae-e51a31bfdcee None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:27.461343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 591/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:27.466145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c731-8e52-4739-89d2-a832e9c7675d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:27.466528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c731-8e52-4739-89d2-a832e9c7675d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:27.466690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c731-8e52-4739-89d2-a832e9c7675d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:27.490128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b775c731-8e52-4739-89d2-a832e9c7675d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:27.491365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b775c731-8e52-4739-89d2-a832e9c7675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:27.499592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 601/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:27.654457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069df86e-54c4-498d-9eae-e51a31bfdcee None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:27.669819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069df86e-54c4-498d-9eae-e51a31bfdcee None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:27.675433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171104bd-e1af-4136-89df-47a8eb66582c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:27.676107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171104bd-e1af-4136-89df-47a8eb66582c service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:27.676410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171104bd-e1af-4136-89df-47a8eb66582c service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:27.676682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171104bd-e1af-4136-89df-47a8eb66582c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:27.684831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171104bd-e1af-4136-89df-47a8eb66582c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:27.686715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171104bd-e1af-4136-89df-47a8eb66582c service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:27.696029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 592/1193] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:28.012855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a82-052c-462a-a9bc-3e6ad1aff6ef None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:28.013054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a82-052c-462a-a9bc-3e6ad1aff6ef None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:28.013318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a82-052c-462a-a9bc-3e6ad1aff6ef None None] PATH_INFO: `/v3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:28.015257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 602/1194] 10.222.0.13 () {54 vars in 925 bytes} [Wed Oct 26 11:08:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:08:28.041818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b299-0f88-4525-926f-73b58b11a529 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:28.042105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b299-0f88-4525-926f-73b58b11a529 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:28.042381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4b299-0f88-4525-926f-73b58b11a529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:28.091868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f5c4b299-0f88-4525-926f-73b58b11a529 None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:28.093329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f5c4b299-0f88-4525-926f-73b58b11a529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:28.163985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 593/1195] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Oct 26 11:08:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:31.985806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b84cc-e5ee-46a9-a12a-e189904f7959 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:31.986239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b84cc-e5ee-46a9-a12a-e189904f7959 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:31.986336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b84cc-e5ee-46a9-a12a-e189904f7959 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:31.987665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 603/1196] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 26 11:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:31.992904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206ed9-9602-4031-9fb1-f10ffc0c0044 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:31.993296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206ed9-9602-4031-9fb1-f10ffc0c0044 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:31.993451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206ed9-9602-4031-9fb1-f10ffc0c0044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:32.013699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fd206ed9-9602-4031-9fb1-f10ffc0c0044 None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:32.016068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fd206ed9-9602-4031-9fb1-f10ffc0c0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:32.024422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 594/1197] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 26 11:08:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:32.032665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b84cc-e5ee-46a9-a12a-e189904f7959 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:32.046478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b84cc-e5ee-46a9-a12a-e189904f7959 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:32.051366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f8cef-8c1f-4789-9204-1a9c7b3a24e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c154983aff14d9895fb4b542db61e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:32.052006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f8cef-8c1f-4789-9204-1a9c7b3a24e0 service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:32.052285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f8cef-8c1f-4789-9204-1a9c7b3a24e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:32.052558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f8cef-8c1f-4789-9204-1a9c7b3a24e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:32.060597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f8cef-8c1f-4789-9204-1a9c7b3a24e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:32.062885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f8cef-8c1f-4789-9204-1a9c7b3a24e0 service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:32.072276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 604/1198] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:39.164878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96524de8-598a-400c-a13e-b89e5b8fdc4e None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:39.165469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96524de8-598a-400c-a13e-b89e5b8fdc4e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:39.165469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96524de8-598a-400c-a13e-b89e5b8fdc4e None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:39.166768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 595/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:39.171584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af2ecd-6260-4b3e-8148-ebcd250cc9be None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:39.172152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af2ecd-6260-4b3e-8148-ebcd250cc9be None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:39.172260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af2ecd-6260-4b3e-8148-ebcd250cc9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:39.193919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f0af2ecd-6260-4b3e-8148-ebcd250cc9be None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:39.195110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f0af2ecd-6260-4b3e-8148-ebcd250cc9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:39.202954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 605/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:08:39.357258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96524de8-598a-400c-a13e-b89e5b8fdc4e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:39.371323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96524de8-598a-400c-a13e-b89e5b8fdc4e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:39.376822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac20ca1-a6c6-4097-b8ad-9b107af7e01b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:39.377411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac20ca1-a6c6-4097-b8ad-9b107af7e01b service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:39.377689 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac20ca1-a6c6-4097-b8ad-9b107af7e01b service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:39.377965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac20ca1-a6c6-4097-b8ad-9b107af7e01b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:39.386446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac20ca1-a6c6-4097-b8ad-9b107af7e01b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:39.388119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac20ca1-a6c6-4097-b8ad-9b107af7e01b service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:39.397862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 596/1201] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:40.453357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed3b67-7a11-4411-bbb0-0db0ef3398db None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:40.453935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed3b67-7a11-4411-bbb0-0db0ef3398db None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:40.453935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed3b67-7a11-4411-bbb0-0db0ef3398db None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:40.455279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 606/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:40.460185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543f54-f90b-4848-a2f8-44793a204715 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:40.460608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543f54-f90b-4848-a2f8-44793a204715 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:40.460734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543f54-f90b-4848-a2f8-44793a204715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:40.484657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b8543f54-f90b-4848-a2f8-44793a204715 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:40.485853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b8543f54-f90b-4848-a2f8-44793a204715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:40.493598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 597/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:40.645277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed3b67-7a11-4411-bbb0-0db0ef3398db None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:40.660184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed3b67-7a11-4411-bbb0-0db0ef3398db None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:40.665659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e5969-b761-4e75-8172-faa3c7df310f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:40.666251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e5969-b761-4e75-8172-faa3c7df310f service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:40.666525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e5969-b761-4e75-8172-faa3c7df310f service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:40.666793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e5969-b761-4e75-8172-faa3c7df310f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:40.675283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e5969-b761-4e75-8172-faa3c7df310f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:40.676882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e5969-b761-4e75-8172-faa3c7df310f service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:40.685918 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 607/1204] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:44.951505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce098b24-5035-40b3-b717-9ec57af70989 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:44.952117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce098b24-5035-40b3-b717-9ec57af70989 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:44.952117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce098b24-5035-40b3-b717-9ec57af70989 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:44.953500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 598/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:44.959455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783416-98c6-413c-9c32-a8ac5b2b48cd None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:44.959926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783416-98c6-413c-9c32-a8ac5b2b48cd None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:44.960093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783416-98c6-413c-9c32-a8ac5b2b48cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:44.979657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f9783416-98c6-413c-9c32-a8ac5b2b48cd None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:44.980865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f9783416-98c6-413c-9c32-a8ac5b2b48cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:44.988917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 608/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:45.279019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce098b24-5035-40b3-b717-9ec57af70989 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:45.280900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce098b24-5035-40b3-b717-9ec57af70989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:45.308658 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:08:45.337573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce098b24-5035-40b3-b717-9ec57af70989 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:45.341131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c0837-577c-4579-840d-427e1f858eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:45.341730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c0837-577c-4579-840d-427e1f858eac service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:45.342041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c0837-577c-4579-840d-427e1f858eac service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:45.342350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c0837-577c-4579-840d-427e1f858eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:45.351068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c0837-577c-4579-840d-427e1f858eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:45.352525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c0837-577c-4579-840d-427e1f858eac service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:45.362268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 599/1207] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:47.527266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c6ba-1629-4810-ad07-e50e58a48714 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:47.527902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c6ba-1629-4810-ad07-e50e58a48714 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:47.527902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c6ba-1629-4810-ad07-e50e58a48714 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:47.529166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 609/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:47.534421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7d9d1-33e8-4379-bcd5-5e447610dca4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:47.534844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7d9d1-33e8-4379-bcd5-5e447610dca4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:47.534978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7d9d1-33e8-4379-bcd5-5e447610dca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:47.554798 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-86f7d9d1-33e8-4379-bcd5-5e447610dca4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:47.558284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-86f7d9d1-33e8-4379-bcd5-5e447610dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:47.564746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 600/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:47.854480 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c6ba-1629-4810-ad07-e50e58a48714 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:47.865266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c6ba-1629-4810-ad07-e50e58a48714 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:47.868714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ff5dc-5825-4f3a-8ec6-7304acbd998c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:47.869306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff5dc-5825-4f3a-8ec6-7304acbd998c service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:47.869581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff5dc-5825-4f3a-8ec6-7304acbd998c service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:47.869849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff5dc-5825-4f3a-8ec6-7304acbd998c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:47.878763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ff5dc-5825-4f3a-8ec6-7304acbd998c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:47.880223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ff5dc-5825-4f3a-8ec6-7304acbd998c service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:47.891895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 610/1210] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:51.592773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eee8c2-4aa0-4618-abda-b00d5685e85a None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:51.593370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eee8c2-4aa0-4618-abda-b00d5685e85a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:51.593370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eee8c2-4aa0-4618-abda-b00d5685e85a None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:51.594671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 601/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:51.600190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb2c5-9aa9-458c-b4f9-173640a21f06 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:51.600625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb2c5-9aa9-458c-b4f9-173640a21f06 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:51.600734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb2c5-9aa9-458c-b4f9-173640a21f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:51.608685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=74 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:08:51.620476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-87acb2c5-9aa9-458c-b4f9-173640a21f06 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:51.622558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-87acb2c5-9aa9-458c-b4f9-173640a21f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:51.630039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 611/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:51.937421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eee8c2-4aa0-4618-abda-b00d5685e85a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:51.939063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b9eee8c2-4aa0-4618-abda-b00d5685e85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:51.995840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eee8c2-4aa0-4618-abda-b00d5685e85a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:51.999271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eca88e-076d-45cc-9673-549309531121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:51.999890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eca88e-076d-45cc-9673-549309531121 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:52.000168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eca88e-076d-45cc-9673-549309531121 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:52.000437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eca88e-076d-45cc-9673-549309531121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:52.009378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eca88e-076d-45cc-9673-549309531121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:52.010788 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eca88e-076d-45cc-9673-549309531121 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:52.020747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 602/1213] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:08:54.172893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1ded6-5f2f-4c0c-aee1-e66ef43926d2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:54.173506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1ded6-5f2f-4c0c-aee1-e66ef43926d2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:54.173506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1ded6-5f2f-4c0c-aee1-e66ef43926d2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:54.174723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 612/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:08:54.180093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2ec3-e9d9-4a86-bcd2-50a1ac8ac6c7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:54.180494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2ec3-e9d9-4a86-bcd2-50a1ac8ac6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:54.180633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2ec3-e9d9-4a86-bcd2-50a1ac8ac6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:54.200246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c0bd2ec3-e9d9-4a86-bcd2-50a1ac8ac6c7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:08:54.201556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c0bd2ec3-e9d9-4a86-bcd2-50a1ac8ac6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:54.209482 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 603/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:08:54.514603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1ded6-5f2f-4c0c-aee1-e66ef43926d2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:08:54.516332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-61f1ded6-5f2f-4c0c-aee1-e66ef43926d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:08:54.529802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1ded6-5f2f-4c0c-aee1-e66ef43926d2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:08:54.533314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50dd69-3c55-4b6a-82b0-7dc87a42ea87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:08:54.533903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50dd69-3c55-4b6a-82b0-7dc87a42ea87 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:08:54.534179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50dd69-3c55-4b6a-82b0-7dc87a42ea87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:08:54.534509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50dd69-3c55-4b6a-82b0-7dc87a42ea87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:08:54.543169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50dd69-3c55-4b6a-82b0-7dc87a42ea87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:08:54.544748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50dd69-3c55-4b6a-82b0-7dc87a42ea87 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:08:54.553975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 613/1216] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:09:04.194888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=47 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:04.550003 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:18.965104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776996-6fce-4ca0-bbe5-b46f0d973b7b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:18.966732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776996-6fce-4ca0-bbe5-b46f0d973b7b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:18.967176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776996-6fce-4ca0-bbe5-b46f0d973b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:18.985090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d853e-ea3c-404a-b853-4d6751dc05bc None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:18.986113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d853e-ea3c-404a-b853-4d6751dc05bc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:18.986601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d853e-ea3c-404a-b853-4d6751dc05bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:18.991927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5e776996-6fce-4ca0-bbe5-b46f0d973b7b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:18.993146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5e776996-6fce-4ca0-bbe5-b46f0d973b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.015477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 604/1217] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.018207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-403d853e-ea3c-404a-b853-4d6751dc05bc None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.019643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-403d853e-ea3c-404a-b853-4d6751dc05bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.026546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f12a-aa30-46e0-ace4-7cfabb6e9b03 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.026844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f12a-aa30-46e0-ace4-7cfabb6e9b03 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.027336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8f12a-aa30-46e0-ace4-7cfabb6e9b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.034129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 614/1218] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:19.051812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca558704-2e3d-4dc4-8abf-c1cc9197b7c6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.051812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca558704-2e3d-4dc4-8abf-c1cc9197b7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.051812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca558704-2e3d-4dc4-8abf-c1cc9197b7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.067623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-41e8f12a-aa30-46e0-ace4-7cfabb6e9b03 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.068962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-41e8f12a-aa30-46e0-ace4-7cfabb6e9b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.088065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ca558704-2e3d-4dc4-8abf-c1cc9197b7c6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.089278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ca558704-2e3d-4dc4-8abf-c1cc9197b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.091385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 605/1219] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.097522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8f12a-aa30-46e0-ace4-7cfabb6e9b03 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.111691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 615/1220] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.111932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8f12a-aa30-46e0-ace4-7cfabb6e9b03 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.116572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca558704-2e3d-4dc4-8abf-c1cc9197b7c6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.123980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.124604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.124878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.125148 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.125801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.126101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.128484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.134291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca558704-2e3d-4dc4-8abf-c1cc9197b7c6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.148656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 606/1221] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:19.154961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.155739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.156041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.156316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.156845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.156974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.157262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.158278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:10:19.158278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 11:10:19.158278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 11:10:19.159177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.178074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68de12-3281-4f52-9484-4e61b18921cf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.185846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 616/1222] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:10:19.193843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cada87-f51c-4b44-91fe-b2d4712f23ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.194477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cada87-f51c-4b44-91fe-b2d4712f23ba admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.194760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cada87-f51c-4b44-91fe-b2d4712f23ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.195030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cada87-f51c-4b44-91fe-b2d4712f23ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.195672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cada87-f51c-4b44-91fe-b2d4712f23ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.197259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.198519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-75cada87-f51c-4b44-91fe-b2d4712f23ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:10:19.198519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 11:10:19.198519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 11:10:19.200323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cada87-f51c-4b44-91fe-b2d4712f23ba admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.218794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b185397-8473-456a-ab35-5be480f22d64 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.228830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6420527-afc8-4a26-a9be-7205ad2ece77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.229443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420527-afc8-4a26-a9be-7205ad2ece77 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.229721 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420527-afc8-4a26-a9be-7205ad2ece77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.229991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420527-afc8-4a26-a9be-7205ad2ece77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.230623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6420527-afc8-4a26-a9be-7205ad2ece77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.237474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6420527-afc8-4a26-a9be-7205ad2ece77 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.355465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 607/1223] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:19.356939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c9981-269d-4462-943e-a5662bfbcb9d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.357233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c9981-269d-4462-943e-a5662bfbcb9d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.357508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c9981-269d-4462-943e-a5662bfbcb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.373875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 617/1224] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/projects => generated 389 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:19.375038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428835d-386b-4e75-8f7d-42271e8e15c1 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.375321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428835d-386b-4e75-8f7d-42271e8e15c1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.375594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428835d-386b-4e75-8f7d-42271e8e15c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.380987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c69c9981-269d-4462-943e-a5662bfbcb9d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.382205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c69c9981-269d-4462-943e-a5662bfbcb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.396453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4428835d-386b-4e75-8f7d-42271e8e15c1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.397622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4428835d-386b-4e75-8f7d-42271e8e15c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.444769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 608/1225] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.446114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e702839-fd52-4859-9501-9b967b898807 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.446405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e702839-fd52-4859-9501-9b967b898807 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.446710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e702839-fd52-4859-9501-9b967b898807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.465357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 618/1226] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.466722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc16f-7539-492a-ab6e-6950c1ed0904 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.467009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc16f-7539-492a-ab6e-6950c1ed0904 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.467117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9e702839-fd52-4859-9501-9b967b898807 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.467284 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc16f-7539-492a-ab6e-6950c1ed0904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.468317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9e702839-fd52-4859-9501-9b967b898807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.476869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 609/1227] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.478029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed6dbf-8946-4dde-811b-ed9a791e88bb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.478316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed6dbf-8946-4dde-811b-ed9a791e88bb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.478592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed6dbf-8946-4dde-811b-ed9a791e88bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.487670 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a0edc16f-7539-492a-ab6e-6950c1ed0904 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.488907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a0edc16f-7539-492a-ab6e-6950c1ed0904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.497531 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 619/1228] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.498765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692e7ec-c266-40fb-b368-81392a117a07 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.499054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692e7ec-c266-40fb-b368-81392a117a07 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.499326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692e7ec-c266-40fb-b368-81392a117a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.500741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c1ed6dbf-8946-4dde-811b-ed9a791e88bb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.501945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c1ed6dbf-8946-4dde-811b-ed9a791e88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.510196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 610/1229] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.510967 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed6dbf-8946-4dde-811b-ed9a791e88bb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.520175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2692e7ec-c266-40fb-b368-81392a117a07 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.521057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed6dbf-8946-4dde-811b-ed9a791e88bb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.521361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2692e7ec-c266-40fb-b368-81392a117a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.525620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb5072-5598-4323-b536-14ad0e6da60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.526217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb5072-5598-4323-b536-14ad0e6da60c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.526497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb5072-5598-4323-b536-14ad0e6da60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.526767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb5072-5598-4323-b536-14ad0e6da60c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.527419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb5072-5598-4323-b536-14ad0e6da60c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.529417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb5072-5598-4323-b536-14ad0e6da60c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.529570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 620/1230] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.531119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692e7ec-c266-40fb-b368-81392a117a07 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.540386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692e7ec-c266-40fb-b368-81392a117a07 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.544973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66237cd-bd7b-4282-85b2-236c53c9e33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.545583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66237cd-bd7b-4282-85b2-236c53c9e33c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.545858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66237cd-bd7b-4282-85b2-236c53c9e33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.546127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66237cd-bd7b-4282-85b2-236c53c9e33c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.546751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66237cd-bd7b-4282-85b2-236c53c9e33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.547807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-d66237cd-bd7b-4282-85b2-236c53c9e33c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:10:19.547807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 11:10:19.547807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 11:10:19.549075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66237cd-bd7b-4282-85b2-236c53c9e33c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.643237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 611/1231] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:19.643237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659be9-f124-4efa-a5a9-afd2f2975df0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.643237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659be9-f124-4efa-a5a9-afd2f2975df0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.643237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659be9-f124-4efa-a5a9-afd2f2975df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.650905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 621/1232] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:19.652195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e6c2-d642-44d1-a6b3-a1c7c1168843 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.652480 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e6c2-d642-44d1-a6b3-a1c7c1168843 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.652754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e6c2-d642-44d1-a6b3-a1c7c1168843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.664748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-31659be9-f124-4efa-a5a9-afd2f2975df0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.665962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-31659be9-f124-4efa-a5a9-afd2f2975df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.671907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f344e6c2-d642-44d1-a6b3-a1c7c1168843 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.673063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f344e6c2-d642-44d1-a6b3-a1c7c1168843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.675188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 612/1233] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.676394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3eca2-c7a1-469e-9d00-70295b1b41f5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.676674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3eca2-c7a1-469e-9d00-70295b1b41f5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.676944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3eca2-c7a1-469e-9d00-70295b1b41f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.681648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 622/1234] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.682957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2023f-97e1-4127-a5b7-3948c595e1f0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.683247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2023f-97e1-4127-a5b7-3948c595e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.683581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2023f-97e1-4127-a5b7-3948c595e1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.699392 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8eb3eca2-c7a1-469e-9d00-70295b1b41f5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.701167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8eb3eca2-c7a1-469e-9d00-70295b1b41f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.704389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-90c2023f-97e1-4127-a5b7-3948c595e1f0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.705585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-90c2023f-97e1-4127-a5b7-3948c595e1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.708981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 613/1235] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.710134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07eaf6-8041-495b-9d2f-c23262ad59ed None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.710417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07eaf6-8041-495b-9d2f-c23262ad59ed None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.710684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07eaf6-8041-495b-9d2f-c23262ad59ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.713518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 623/1236] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.714687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e9478-3e69-4d9c-a676-2c96de0d0a6e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.714973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e9478-3e69-4d9c-a676-2c96de0d0a6e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.715248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e9478-3e69-4d9c-a676-2c96de0d0a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.734520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3a07eaf6-8041-495b-9d2f-c23262ad59ed None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.735640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-019e9478-3e69-4d9c-a676-2c96de0d0a6e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:19.735774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3a07eaf6-8041-495b-9d2f-c23262ad59ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.736824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-019e9478-3e69-4d9c-a676-2c96de0d0a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:19.743594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 614/1237] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.744399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07eaf6-8041-495b-9d2f-c23262ad59ed None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.744617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 624/1238] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:19.745384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e9478-3e69-4d9c-a676-2c96de0d0a6e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.755040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07eaf6-8041-495b-9d2f-c23262ad59ed None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.756889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e9478-3e69-4d9c-a676-2c96de0d0a6e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.759655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.760284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.760567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.760848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.761491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.761613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.762345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.762755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.763045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.763265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.763680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.765997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.773402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 615/1239] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:19.774195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.780378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 625/1240] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:19.781209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.784773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2920360-f967-4225-b542-7e89da9b3649 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.789394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.789979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.790259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.790562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.791199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.791312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d1cc8-2329-4188-9dd4-6d01ce93dd90 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.791499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.793127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.796444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.797028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.797307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.797580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.798215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.798518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.800163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.805955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 616/1241] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:19.806860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.813851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 626/1242] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:19.814683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.817869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af20e05-b223-4b37-a32a-06416bb98059 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.822758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.823352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.823629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.823929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.824484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431752f0-aade-4449-bcff-1729c06651e4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.824613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.825110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.826744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.829221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.829801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.830086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.830364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.830990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.831287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.832873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.838671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 617/1243] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:19.839529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.846810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 627/1244] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:19.846810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.855422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4f410-7271-4f00-a428-912f5b5a92c8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.858226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625ca76-0423-41c4-abe3-1fd1330a6078 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.860770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.861469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.861869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.862151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.862792 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.862895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.863216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.863451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.863729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.864033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.864686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:19.864848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.864989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.866566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.877766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 628/1245] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:19.878885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.879775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 618/1246] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:19.880599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.890373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2974316-a39c-45af-8968-adfa6d190897 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.895173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.895742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.896058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.896329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.896969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.898568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.900543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273653-c6e0-4b8c-ae52-1768b2a1acd0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.903870 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:19.905742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.906352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.906632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.906907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.907542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.911952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 619/1247] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:19.912081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.912214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.914660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:19.923073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 629/1248] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:19.923073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:19.923926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940b3fd-5454-4df3-af86-fbacec7d2f85 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.926911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.927514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.927811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.928095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.928725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.930631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:19.931147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfe9cd-bc08-449d-a8f8-69ce52b82dcb admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:19.936735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:19.937364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:19.937631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:19.937901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:19.938524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:19.940441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.023527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 620/1249] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.024306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.040750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 630/1250] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:19 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.041567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.093868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4b801-d38e-4f9f-bb47-c499ec91d370 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.107351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20e860-2477-47e0-a396-51ce283a512d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.112179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.112831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.113108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.113377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.114051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.116004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.152320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.152320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.152320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.152320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.154337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.214450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 631/1251] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.214450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.240781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 621/1252] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.241524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.276657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691b24c-4977-4075-9874-9d4d47de9262 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.306508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda620d2-6c81-43cf-af6f-f7f98d911239 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.311569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.312230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.312515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.312784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.313422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.315357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.331925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:20.332575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.333313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.333593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.333896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.334517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.336471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.351888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:20.411872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 622/1253] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/projects => generated 389 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.412749 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.442508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 632/1254] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.442508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.480548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0bc32-5f16-44af-bb80-131164866822 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.515770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9530cb-797e-4077-a8a3-45d05589a67a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.520591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.521251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.521531 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.521823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] PATH_INFO: `/v3/projects/6b165f81d6714550af6432b58e3af800/users/81b22359bd1c472090b8d12c6f192102/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.537067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b165f81d6714550af6432b58e3af800, user_id=81b22359bd1c472090b8d12c6f192102, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.539694 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.544597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.547655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.547655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.547655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] PATH_INFO: `/v3/projects/b68088f7836d42eab080dd159a453ca9/users/f4c15ef6162348f9b1801c15b959b733/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.560162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68088f7836d42eab080dd159a453ca9, user_id=f4c15ef6162348f9b1801c15b959b733, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.562798 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.627530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 633/1255] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:20 2022] PUT /identity/v3/projects/6b165f81d6714550af6432b58e3af800/users/81b22359bd1c472090b8d12c6f192102/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:20.628624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.656907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 623/1256] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:20 2022] PUT /identity/v3/projects/b68088f7836d42eab080dd159a453ca9/users/f4c15ef6162348f9b1801c15b959b733/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:20.657840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.692148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252b578-63fe-472e-bc95-dcd67c00968d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.721306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c834d1-4b0e-4e43-a454-3fcb48eb91d8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.726011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.726621 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.726898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.727170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.727854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.729793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.748891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.753873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.753873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.753873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.753873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.753873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.833804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 624/1257] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.834554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.844283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1035ae-82af-4da8-ae38-fbce13cd7762 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.845819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 634/1258] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.846638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.849911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.850507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.850788 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.851083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.851721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.853678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.856651 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc2e19-1059-4db4-9832-59d0da056b67 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.861437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.866098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.866098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.866098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.866098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.866098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.936507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:20.942344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 625/1259] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:20.943108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:20.953591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cd268-e71d-44cc-b4a2-fdb0f22e5214 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:20.958093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f6fcc-0edf-4587-b441-8850b9d829e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:20.973678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f6fcc-0edf-4587-b441-8850b9d829e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:20.973678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f6fcc-0edf-4587-b441-8850b9d829e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:20.973678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f6fcc-0edf-4587-b441-8850b9d829e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:20.973678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f6fcc-0edf-4587-b441-8850b9d829e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:20.973678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f6fcc-0edf-4587-b441-8850b9d829e5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:20.993798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:21.000382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 635/1260] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:21.001142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.010926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f99776-031d-40e4-90b5-74a6cd30f353 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.016366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786ac3d-bb4a-4fc0-aa81-6700dcf596d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.016974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786ac3d-bb4a-4fc0-aa81-6700dcf596d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.017256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786ac3d-bb4a-4fc0-aa81-6700dcf596d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.017584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786ac3d-bb4a-4fc0-aa81-6700dcf596d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.018211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786ac3d-bb4a-4fc0-aa81-6700dcf596d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.020186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786ac3d-bb4a-4fc0-aa81-6700dcf596d1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.085292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 626/1261] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:20 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:21.086660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0039f7-55fb-442f-bf69-429af0510fe0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.086941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0039f7-55fb-442f-bf69-429af0510fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.087211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0039f7-55fb-442f-bf69-429af0510fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.114265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 636/1262] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:21 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:21.115636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28957441-d99e-4cf3-8763-e06aac933df8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.117273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6a0039f7-55fb-442f-bf69-429af0510fe0 None None] MFA Rules not processed for user `81b22359bd1c472090b8d12c6f192102`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:21.117404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28957441-d99e-4cf3-8763-e06aac933df8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.117404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28957441-d99e-4cf3-8763-e06aac933df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.117581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6a0039f7-55fb-442f-bf69-429af0510fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:21.136665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-28957441-d99e-4cf3-8763-e06aac933df8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:21.138565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-28957441-d99e-4cf3-8763-e06aac933df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:21.146137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 637/1263] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:21.146915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28957441-d99e-4cf3-8763-e06aac933df8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.156068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28957441-d99e-4cf3-8763-e06aac933df8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.160666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.163023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.163023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.163023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.163023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.164021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.177540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 638/1264] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.177540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.185373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ad4f3-924f-4376-9d80-8fed56a5ab77 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.189866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.190436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.190711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.190978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.191371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 627/1265] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:21.191599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.192238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0039f7-55fb-442f-bf69-429af0510fe0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.193191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.202758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0039f7-55fb-442f-bf69-429af0510fe0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.203610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 639/1266] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.207235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.208303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.209076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.209356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.209627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.210300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.211950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.218163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3d3da-9af3-4c63-8918-516134d781ef admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.222826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 628/1267] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.223552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.223719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.224277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.224614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.224934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.225892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.227366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.233676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211db165-0c02-4201-8f53-74840c052492 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.238300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.238703 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 640/1268] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.238884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.239165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.239435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.239533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.240095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.241687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.251182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79fcbc-f4c6-421b-9a1f-6f29020448e0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.255558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 629/1269] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.255558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.255976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.256561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.256836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.257104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.257726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.259281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.269596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 641/1270] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.270279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.279885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada86b8-c0a8-4fe4-ae66-6141134d538f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.284382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.284954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.285225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.285490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.286091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.287631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.298442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 642/1271] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.299365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.314635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef436c7-a557-4116-8f85-45e9f0353690 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.318220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.318819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.319094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.319363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.328544 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.329994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.340393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 630/1272] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:21.341095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.350166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80a1a-2937-4117-abd3-8eb6ac7b23a0 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.354911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.355492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.356362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.356634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.357262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.357920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada8d8f-69fd-4ad0-b97a-979738cf438e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.358866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.361723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.362338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.362619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.362893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.369221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 631/1273] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.372219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.373754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.373999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.383244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 643/1274] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:21.383569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867a2b-96c1-4877-8300-8f6839e668f5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.384257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.388690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.389276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.389552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.389823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.390443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.392059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.394508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af789d-8de2-42e7-8424-6fc0dc703153 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.400238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.400889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.401179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.401678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.402081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.403855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.405114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 632/1275] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.405746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.415143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4935a0-cf66-4335-95db-bc37a3d4f5e0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.421907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 644/1276] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.421907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.422113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.422113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.422113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.422113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.422453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.424152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.429592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2ad58-d7f6-4b34-8129-910e7f470cab admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.438382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.438559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 633/1277] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.438646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.438721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.438812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.438812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.438812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.438812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.446098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c959efd-062a-41fb-a68d-60db6f2fa015 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.450516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 645/1278] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:21.458026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.458169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.458169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.458169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.458169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] PATH_INFO: `/v3/projects/0485207d86e243cf8e0edf419b5dfdb6/users/eca696f5dfaa466fa06782c82ea42758/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.462288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3a614-4e1d-434e-a690-661453307110 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.467258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.468228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.468425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.468706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] PATH_INFO: `/v3/projects/5fa962ac43104bbfa82b0ce7693ec0b4/users/d8c0712efd3942489b4469d4ac54b7a2/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.471509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0485207d86e243cf8e0edf419b5dfdb6, user_id=eca696f5dfaa466fa06782c82ea42758, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.474293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.478028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:21.487687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa962ac43104bbfa82b0ce7693ec0b4, user_id=d8c0712efd3942489b4469d4ac54b7a2, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.490411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.569133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 646/1280] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:21 2022] PUT /identity/v3/projects/5fa962ac43104bbfa82b0ce7693ec0b4/users/d8c0712efd3942489b4469d4ac54b7a2/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:21.570293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.580533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 634/1280] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:21 2022] PUT /identity/v3/projects/0485207d86e243cf8e0edf419b5dfdb6/users/eca696f5dfaa466fa06782c82ea42758/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:21.582265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.618986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:21.644367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567227e2-7c75-421c-84c5-ebb6cf6e5698 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.649550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.650225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.650506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.650779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] PATH_INFO: `/v3/projects/cd0fd618c6824a0aad3818cb35edf2ce/users/04dd6b2d2aad43c4b3eb44dfa0a92fd3/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.656790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31b154-f469-4392-b6e6-bf1caff8b04e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.661969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.662638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.662920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.663195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] PATH_INFO: `/v3/projects/a490c158ed0542fa863440b35aa0d44d/users/bfa99ee8ae73487da15a4adf72c473cb/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.674503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd0fd618c6824a0aad3818cb35edf2ce, user_id=04dd6b2d2aad43c4b3eb44dfa0a92fd3, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.678076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.680284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a490c158ed0542fa863440b35aa0d44d, user_id=bfa99ee8ae73487da15a4adf72c473cb, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.683127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.801096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 635/1281] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:21 2022] PUT /identity/v3/projects/a490c158ed0542fa863440b35aa0d44d/users/bfa99ee8ae73487da15a4adf72c473cb/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:21.801310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 647/1282] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:21 2022] PUT /identity/v3/projects/cd0fd618c6824a0aad3818cb35edf2ce/users/04dd6b2d2aad43c4b3eb44dfa0a92fd3/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:21.803830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.803992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.870136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518266-3ba2-4a41-a584-38d6b1853826 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.870394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29711e-65c6-4b5c-8ef6-f83f8fb9e5ec admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:21.875031 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf3ec8-799e-4f96-9db7-f114742d8a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.875200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1029420-c495-4efb-9aae-e02a4f709ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:21.875890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1029420-c495-4efb-9aae-e02a4f709ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.876241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1029420-c495-4efb-9aae-e02a4f709ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.876584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1029420-c495-4efb-9aae-e02a4f709ab5 admin admin] PATH_INFO: `/v3/projects/14a7a0c0ef824bbd928dd8a18e41b65e/users/83e3e4b159c94fdf9a03b72ebf62ac46/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.876690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf3ec8-799e-4f96-9db7-f114742d8a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.877003 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf3ec8-799e-4f96-9db7-f114742d8a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.877350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf3ec8-799e-4f96-9db7-f114742d8a8b admin admin] PATH_INFO: `/v3/projects/f87286f4fab94c6192ee325fe6c75315/users/95eef108b14a4a90a8b617088c904a08/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.895197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf3ec8-799e-4f96-9db7-f114742d8a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87286f4fab94c6192ee325fe6c75315, user_id=95eef108b14a4a90a8b617088c904a08, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.895334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1029420-c495-4efb-9aae-e02a4f709ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a7a0c0ef824bbd928dd8a18e41b65e, user_id=83e3e4b159c94fdf9a03b72ebf62ac46, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:21.897412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf3ec8-799e-4f96-9db7-f114742d8a8b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.898081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1029420-c495-4efb-9aae-e02a4f709ab5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:21.982299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 648/1283] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:21 2022] PUT /identity/v3/projects/f87286f4fab94c6192ee325fe6c75315/users/95eef108b14a4a90a8b617088c904a08/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:21.983883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a7809-fd70-4a13-979b-45daf7d343f9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.984195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a7809-fd70-4a13-979b-45daf7d343f9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.984467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a7809-fd70-4a13-979b-45daf7d343f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.003454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fa9a7809-fd70-4a13-979b-45daf7d343f9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:22.004782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fa9a7809-fd70-4a13-979b-45daf7d343f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.014946 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 636/1284] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:21 2022] PUT /identity/v3/projects/14a7a0c0ef824bbd928dd8a18e41b65e/users/83e3e4b159c94fdf9a03b72ebf62ac46/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:22.016269 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b2456-bd65-481b-bafc-8356e64e642d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.016623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b2456-bd65-481b-bafc-8356e64e642d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.016964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b2456-bd65-481b-bafc-8356e64e642d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.058381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5b8b2456-bd65-481b-bafc-8356e64e642d None None] MFA Rules not processed for user `d8c0712efd3942489b4469d4ac54b7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:22.059768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5b8b2456-bd65-481b-bafc-8356e64e642d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.082748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 649/1285] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:22.084044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c1fc5-2216-47a2-ba98-1aeb1ab2fa24 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.084334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c1fc5-2216-47a2-ba98-1aeb1ab2fa24 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.084610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c1fc5-2216-47a2-ba98-1aeb1ab2fa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.104642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-830c1fc5-2216-47a2-ba98-1aeb1ab2fa24 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:22.105935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-830c1fc5-2216-47a2-ba98-1aeb1ab2fa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.112019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:22.113972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 650/1286] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:22.115165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb791c72-f874-4410-a042-b2193013fb96 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.115680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb791c72-f874-4410-a042-b2193013fb96 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.115680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb791c72-f874-4410-a042-b2193013fb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.133055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 637/1287] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:22.134685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a257ef-cb26-4854-811d-ac7b138b29f4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.135249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a257ef-cb26-4854-811d-ac7b138b29f4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.138132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a257ef-cb26-4854-811d-ac7b138b29f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.148568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fb791c72-f874-4410-a042-b2193013fb96 None None] MFA Rules not processed for user `04dd6b2d2aad43c4b3eb44dfa0a92fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:22.149998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fb791c72-f874-4410-a042-b2193013fb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.166852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a0a257ef-cb26-4854-811d-ac7b138b29f4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:22.169064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a0a257ef-cb26-4854-811d-ac7b138b29f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.181810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 638/1288] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:22.181810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9a0b-373d-4e10-b9b0-c51654f70af4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.181810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9a0b-373d-4e10-b9b0-c51654f70af4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.182184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:22.182290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9a0b-373d-4e10-b9b0-c51654f70af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.217321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-44cb9a0b-373d-4e10-b9b0-c51654f70af4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:22.218538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-44cb9a0b-373d-4e10-b9b0-c51654f70af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.236114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 639/1289] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:22.237717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9a0b-373d-4e10-b9b0-c51654f70af4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.239624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 651/1290] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:22.245068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb791c72-f874-4410-a042-b2193013fb96 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.318108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb791c72-f874-4410-a042-b2193013fb96 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.322216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbec4e-17eb-410b-9d11-e30f52e03865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.323060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbec4e-17eb-410b-9d11-e30f52e03865 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.323509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbec4e-17eb-410b-9d11-e30f52e03865 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.323969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbec4e-17eb-410b-9d11-e30f52e03865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.335713 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9a0b-373d-4e10-b9b0-c51654f70af4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.336514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbec4e-17eb-410b-9d11-e30f52e03865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.337963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbec4e-17eb-410b-9d11-e30f52e03865 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.340140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.341071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.341629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.342260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.348720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 652/1291] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:22.350006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae2cec-82b8-4b4a-b03f-2351135bfe5a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.350330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae2cec-82b8-4b4a-b03f-2351135bfe5a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.350615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae2cec-82b8-4b4a-b03f-2351135bfe5a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.356177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.357583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 653/1292] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 26 11:10:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:22.358111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae2cec-82b8-4b4a-b03f-2351135bfe5a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.359195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.359898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e2ae2cec-82b8-4b4a-b03f-2351135bfe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.372219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 640/1293] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:22.378616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.378768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae2cec-82b8-4b4a-b03f-2351135bfe5a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.382749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.383459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.384044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.384253 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d0970-216d-42c5-aed9-353d156b9187 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.384543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.389949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c22eda-4ba4-4f16-8fff-f6da162e7d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.390554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c22eda-4ba4-4f16-8fff-f6da162e7d8b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.390832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c22eda-4ba4-4f16-8fff-f6da162e7d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.391123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c22eda-4ba4-4f16-8fff-f6da162e7d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.396247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.398100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.409901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 654/1294] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:22.410779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.412984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c22eda-4ba4-4f16-8fff-f6da162e7d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.415677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c22eda-4ba4-4f16-8fff-f6da162e7d8b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.420484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a18cc-8a18-407b-bf92-9d2374a24510 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.426497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.427283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.427886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.428023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 641/1295] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:22.428598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.429264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59395ccd-d19e-4d9b-a83d-f805865e1001 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.429392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.429523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59395ccd-d19e-4d9b-a83d-f805865e1001 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.430148 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59395ccd-d19e-4d9b-a83d-f805865e1001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.431226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.475806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-59395ccd-d19e-4d9b-a83d-f805865e1001 None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:22.477092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-59395ccd-d19e-4d9b-a83d-f805865e1001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.516752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 655/1296] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:22.520906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.613944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 642/1297] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:22.624229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59395ccd-d19e-4d9b-a83d-f805865e1001 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.645790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59395ccd-d19e-4d9b-a83d-f805865e1001 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.658874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b9e4-6acd-4297-8272-f9b6451bc02a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.664717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.666266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.666420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.666741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.667384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.670680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.714326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:22.747135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.749026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.749026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.749026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.766246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 656/1298] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:22 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:22.770208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.780386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b0cbb-5e88-4412-b6f4-962800b40920 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.785453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.787506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.787966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.788321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.789239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.792176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.805565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 657/1299] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:22.810348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.819462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.820986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.821103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caa2f9-c5b1-48c2-be60-dc770f9c083e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.826553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.827222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.827822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.827822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] PATH_INFO: `/v3/projects/fbc98fec4454406d9d92aebf00a538f7/users/097837e49e1d4516a04a74846284ad0d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.831096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 643/1300] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:22.860376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:22.885613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc98fec4454406d9d92aebf00a538f7, user_id=097837e49e1d4516a04a74846284ad0d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.885613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.963622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 658/1301] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:22 2022] PUT /identity/v3/projects/fbc98fec4454406d9d92aebf00a538f7/users/097837e49e1d4516a04a74846284ad0d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:22.968201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:22.985820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b70331-2d7b-4a50-9ffd-b8ea5f5936f8 service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:23.115048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57888e-897d-4b6f-976e-c05b7ab431b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:23.115719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57888e-897d-4b6f-976e-c05b7ab431b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:23.116463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57888e-897d-4b6f-976e-c05b7ab431b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:23.116835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57888e-897d-4b6f-976e-c05b7ab431b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:23.122275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57888e-897d-4b6f-976e-c05b7ab431b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:23.124396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57888e-897d-4b6f-976e-c05b7ab431b4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:23.141881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:23.153691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 644/1302] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:23.158986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d79c2a-6de8-4fa0-af7f-853801ed49a3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:23.172730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fee450-743a-4268-a100-d5993da5ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:23.173406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fee450-743a-4268-a100-d5993da5ff89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:23.173682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fee450-743a-4268-a100-d5993da5ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:23.173952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fee450-743a-4268-a100-d5993da5ff89 admin admin] PATH_INFO: `/v3/projects/fbc98fec4454406d9d92aebf00a538f7/users/097837e49e1d4516a04a74846284ad0d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:23.179629 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fee450-743a-4268-a100-d5993da5ff89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc98fec4454406d9d92aebf00a538f7, user_id=097837e49e1d4516a04a74846284ad0d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:23.187863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fee450-743a-4268-a100-d5993da5ff89 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:23.188644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2124ef-a425-4eb5-8f57-f460b4cb4948 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:23.189148 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2124ef-a425-4eb5-8f57-f460b4cb4948 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:23.189649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2124ef-a425-4eb5-8f57-f460b4cb4948 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:23.191662 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 645/1303] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 26 11:10:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:23.202726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b32cf9-093e-41c1-a018-e54d9caa7f79 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:23.211869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 659/1304] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:23 2022] PUT /identity/v3/projects/fbc98fec4454406d9d92aebf00a538f7/users/097837e49e1d4516a04a74846284ad0d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:23.213404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b32cf9-093e-41c1-a018-e54d9caa7f79 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:23.214109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b32cf9-093e-41c1-a018-e54d9caa7f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:23.214224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66891c-a9a0-4c09-b875-e40ad59a5f3e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:23.215958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66891c-a9a0-4c09-b875-e40ad59a5f3e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:23.216271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66891c-a9a0-4c09-b875-e40ad59a5f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:23.255156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1a66891c-a9a0-4c09-b875-e40ad59a5f3e None None] MFA Rules not processed for user `097837e49e1d4516a04a74846284ad0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:23.257318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1a66891c-a9a0-4c09-b875-e40ad59a5f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:23.265975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-14b32cf9-093e-41c1-a018-e54d9caa7f79 None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:23.267297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-14b32cf9-093e-41c1-a018-e54d9caa7f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:23.369638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 660/1305] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:23.390154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66891c-a9a0-4c09-b875-e40ad59a5f3e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:23.431414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 646/1306] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:23.483761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b32cf9-093e-41c1-a018-e54d9caa7f79 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:23.492807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:23.501038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66891c-a9a0-4c09-b875-e40ad59a5f3e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:23.513026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:23.513662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:23.513943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:23.514218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:23.534409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:23.536380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:23.559948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 661/1307] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:10:23.569812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b32cf9-093e-41c1-a018-e54d9caa7f79 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:23.576061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043bf1d-6853-4859-a1e7-194660738243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:23.576838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043bf1d-6853-4859-a1e7-194660738243 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:23.577137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043bf1d-6853-4859-a1e7-194660738243 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:23.577394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043bf1d-6853-4859-a1e7-194660738243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:23.594520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043bf1d-6853-4859-a1e7-194660738243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:23.596487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043bf1d-6853-4859-a1e7-194660738243 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:23.625433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 647/1308] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:24.926407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:24.929418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21ce4e-4322-4a2c-85a2-b2abf30b8c57 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:24.929916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21ce4e-4322-4a2c-85a2-b2abf30b8c57 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:24.930013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21ce4e-4322-4a2c-85a2-b2abf30b8c57 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:24.931302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 648/1309] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 26 11:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:24.939833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc7aea-1d9f-4dea-9f58-a8fba232149a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:24.939833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc7aea-1d9f-4dea-9f58-a8fba232149a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:24.940077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc7aea-1d9f-4dea-9f58-a8fba232149a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:24.992637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-28dc7aea-1d9f-4dea-9f58-a8fba232149a None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:24.994671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-28dc7aea-1d9f-4dea-9f58-a8fba232149a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:25.007550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 649/1310] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 26 11:10:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:25.013968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1a04-2a10-45f3-add7-5d615f0015ee None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:25.014260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1a04-2a10-45f3-add7-5d615f0015ee None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:25.014499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1a04-2a10-45f3-add7-5d615f0015ee None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:25.016059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 650/1311] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 26 11:10:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:10:25.036754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38dfe7-3c10-49a5-8025-d05dd03e4355 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:25.037583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38dfe7-3c10-49a5-8025-d05dd03e4355 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:25.038209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38dfe7-3c10-49a5-8025-d05dd03e4355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:25.072416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68be3b-d242-4415-921d-3cc4d4f3f1f5 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:25.078848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:25.079667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:25.080081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:25.080456 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] PATH_INFO: `/v3/users/04dd6b2d2aad43c4b3eb44dfa0a92fd3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:25.083283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04dd6b2d2aad43c4b3eb44dfa0a92fd3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:25.087310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:25.091388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:25.093547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4a38dfe7-3c10-49a5-8025-d05dd03e4355 None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:25.094802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4a38dfe7-3c10-49a5-8025-d05dd03e4355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:25.122776 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 651/1312] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:25.138011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38dfe7-3c10-49a5-8025-d05dd03e4355 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:25.150477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38dfe7-3c10-49a5-8025-d05dd03e4355 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:25.155922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e266377-98cb-4554-b3d1-3fd088da3f15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:25.163017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266377-98cb-4554-b3d1-3fd088da3f15 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:25.163017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266377-98cb-4554-b3d1-3fd088da3f15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:25.163017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e266377-98cb-4554-b3d1-3fd088da3f15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:25.176672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e266377-98cb-4554-b3d1-3fd088da3f15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:25.178235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e266377-98cb-4554-b3d1-3fd088da3f15 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:25.191390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 652/1313] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:10:25.378902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04dd6b2d2aad43c4b3eb44dfa0a92fd3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:25.379950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04dd6b2d2aad43c4b3eb44dfa0a92fd3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:25.484616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04dd6b2d2aad43c4b3eb44dfa0a92fd3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:25.485261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04dd6b2d2aad43c4b3eb44dfa0a92fd3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:25.493946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04dd6b2d2aad43c4b3eb44dfa0a92fd3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:25.524065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 662/1314] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:24 2022] DELETE /identity/v3/users/04dd6b2d2aad43c4b3eb44dfa0a92fd3 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:25.532748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:25.532748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:25.532748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:25.567725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:25.569577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:25.690614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 653/1315] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:25.729707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:25.731764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:25.803018 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e7bda-560a-4adf-9a32-b96e1aca5d2b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:25.809514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:25.810159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:25.810440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:25.810717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:25.821579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:25.823030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:25.852182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 663/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:27.345415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:27.346899 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:27.369293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c44d3-1ba8-48bb-b69e-8bf2d04a38ef None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:27.385925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:27.386584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:27.387398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:27.388373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] PATH_INFO: `/v3/projects/cd0fd618c6824a0aad3818cb35edf2ce` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:27.388953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd0fd618c6824a0aad3818cb35edf2ce)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:27.390952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:27.612976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0fd618c6824a0aad3818cb35edf2ce', 'request_id': 'req-85c72c41-a556-4302-9e8e-8038b65a7e98'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:27.631267 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0fd618c6824a0aad3818cb35edf2ce', 'request_id': 'req-85c72c41-a556-4302-9e8e-8038b65a7e98'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:27.647700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0fd618c6824a0aad3818cb35edf2ce', 'request_id': 'req-85c72c41-a556-4302-9e8e-8038b65a7e98'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:27.740275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:27.741990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:27.748976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] The token cache is being invalidate because project cd0fd618c6824a0aad3818cb35edf2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:27.756401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:27.756401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:27.756401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 654/1317] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:27 2022] DELETE /identity/v3/projects/cd0fd618c6824a0aad3818cb35edf2ce => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:27.759257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:27.762769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:27.766228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea8cbe-ba85-4a35-8ffe-64916b139102 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:27.789761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:27.867128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:27.868011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:27.868575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:27.869285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] PATH_INFO: `/v3/users/d8c0712efd3942489b4469d4ac54b7a2` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:27.870853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c0712efd3942489b4469d4ac54b7a2)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:27.874735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:27.935267 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c72c41-a556-4302-9e8e-8038b65a7e98 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:27.940659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:27.941548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:27.941964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:27.942247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] PATH_INFO: `/v3/users/097837e49e1d4516a04a74846284ad0d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:27.943821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=097837e49e1d4516a04a74846284ad0d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:27.945002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: WARNING py.warnings [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:10:27.945002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: warnings.warn(msg) Oct 26 11:10:27.945002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: Oct 26 11:10:27.946530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:27.989612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:28.176112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c0712efd3942489b4469d4ac54b7a2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.176112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c0712efd3942489b4469d4ac54b7a2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.268263 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c0712efd3942489b4469d4ac54b7a2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.269415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c0712efd3942489b4469d4ac54b7a2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.279853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4ccc35ce-f540-4c92-b121-f23d60ce9564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c0712efd3942489b4469d4ac54b7a2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.347482 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097837e49e1d4516a04a74846284ad0d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.366325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 664/1318] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:27 2022] DELETE /identity/v3/users/d8c0712efd3942489b4469d4ac54b7a2 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:28.371636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:28.380830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:28.380830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:28.397246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097837e49e1d4516a04a74846284ad0d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.410316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097837e49e1d4516a04a74846284ad0d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.451819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:28.453144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:28.455956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097837e49e1d4516a04a74846284ad0d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.457896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097837e49e1d4516a04a74846284ad0d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:28.553872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 665/1319] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:28.563822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 655/1320] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:27 2022] DELETE /identity/v3/users/097837e49e1d4516a04a74846284ad0d => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:28.589431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:28.592596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:28.761429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033986-acbc-4b4c-a858-4719e02103d5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:28.769478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba58fd-337e-4275-a366-92f447be4fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:28.769991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba58fd-337e-4275-a366-92f447be4fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:28.770337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba58fd-337e-4275-a366-92f447be4fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:28.770628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba58fd-337e-4275-a366-92f447be4fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:28.898520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba58fd-337e-4275-a366-92f447be4fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:28.899213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba58fd-337e-4275-a366-92f447be4fd2 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:28.917006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 656/1321] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:29.330888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:29.975955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:29.978503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:30.015838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfd9d1-4ab2-41e8-9248-47c1f8f529a0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:30.032994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:30.035512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:30.036460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:30.038431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] PATH_INFO: `/v3/projects/fbc98fec4454406d9d92aebf00a538f7` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:30.044068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc98fec4454406d9d92aebf00a538f7)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:30.045747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:10:30.045747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Oct 26 11:10:30.045747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: Oct 26 11:10:30.074655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:30.333369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc98fec4454406d9d92aebf00a538f7', 'request_id': 'req-1f0d2ecb-1971-460a-bd51-e72af3da7724'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:30.352563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc98fec4454406d9d92aebf00a538f7', 'request_id': 'req-1f0d2ecb-1971-460a-bd51-e72af3da7724'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:30.370143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc98fec4454406d9d92aebf00a538f7', 'request_id': 'req-1f0d2ecb-1971-460a-bd51-e72af3da7724'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:30.507342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] The token cache is being invalidate because project fbc98fec4454406d9d92aebf00a538f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:30.507926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:30.510170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:30.519610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 666/1322] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:29 2022] DELETE /identity/v3/projects/fbc98fec4454406d9d92aebf00a538f7 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:30.520863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287ac5-9304-49e7-a41e-9963809e37d1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:30.521366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287ac5-9304-49e7-a41e-9963809e37d1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:30.521461 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c287ac5-9304-49e7-a41e-9963809e37d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:30.556828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:30.558683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:30.577479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7c287ac5-9304-49e7-a41e-9963809e37d1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:30.578755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7c287ac5-9304-49e7-a41e-9963809e37d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:30.754509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:30.822680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 657/1323] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:30.832670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eef110-1aef-4166-9136-210c8f2a7157 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:30.833203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eef110-1aef-4166-9136-210c8f2a7157 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:30.833633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eef110-1aef-4166-9136-210c8f2a7157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:30.879358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d2ecb-1971-460a-bd51-e72af3da7724 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:30.880958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a1eef110-1aef-4166-9136-210c8f2a7157 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:30.882806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a1eef110-1aef-4166-9136-210c8f2a7157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:30.898165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 658/1324] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:30.905920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:30.906844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:30.907186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:30.907562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] PATH_INFO: `/v3/projects/5fa962ac43104bbfa82b0ce7693ec0b4` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:30.910912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eef110-1aef-4166-9136-210c8f2a7157 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:30.914799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa962ac43104bbfa82b0ce7693ec0b4)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:30.918150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:30.955372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eef110-1aef-4166-9136-210c8f2a7157 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:30.986459 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:30.986898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:30.988324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:30.988324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:30.991313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:30.991634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:30.993359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.031115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 659/1325] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:31.034335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.062047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f9ab-c607-4531-a1c5-26d17c177b55 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.085987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.086625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.086908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.087185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.087971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.090378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.156400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa962ac43104bbfa82b0ce7693ec0b4', 'request_id': 'req-543cc03e-5fd0-4253-9289-fd12c683249a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:31.166941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa962ac43104bbfa82b0ce7693ec0b4', 'request_id': 'req-543cc03e-5fd0-4253-9289-fd12c683249a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:31.191537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa962ac43104bbfa82b0ce7693ec0b4', 'request_id': 'req-543cc03e-5fd0-4253-9289-fd12c683249a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:31.270263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 660/1326] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:31 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:10:31.274114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.292119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] The token cache is being invalidate because project 5fa962ac43104bbfa82b0ce7693ec0b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:31.301000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:31.301000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-543cc03e-5fd0-4253-9289-fd12c683249a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:31.301000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 667/1327] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:30 2022] DELETE /identity/v3/projects/5fa962ac43104bbfa82b0ce7693ec0b4 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:31.305611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b00ea2-54da-45db-8ba3-c00b69be4c91 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.305899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b00ea2-54da-45db-8ba3-c00b69be4c91 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.306176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b00ea2-54da-45db-8ba3-c00b69be4c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.356855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-89b00ea2-54da-45db-8ba3-c00b69be4c91 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:31.358093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-89b00ea2-54da-45db-8ba3-c00b69be4c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:31.437408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5d33f-67fd-4d24-96a3-5e799a38798c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.454631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.455268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.455546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.455835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.456479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.458452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.506817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 668/1328] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:31.509907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea80b8-a2d2-4c03-9a88-3a98d778085b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.513088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea80b8-a2d2-4c03-9a88-3a98d778085b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.518643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea80b8-a2d2-4c03-9a88-3a98d778085b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.562813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b1ea80b8-a2d2-4c03-9a88-3a98d778085b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:31.583164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b1ea80b8-a2d2-4c03-9a88-3a98d778085b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:31.621361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 669/1329] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:31.624081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea80b8-a2d2-4c03-9a88-3a98d778085b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.646873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea80b8-a2d2-4c03-9a88-3a98d778085b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.662617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.662893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.663141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.663534 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.664834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:31.665215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.669963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.676856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 661/1330] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:31 2022] POST /identity/v3/users => generated 294 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:31.683513 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.688387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:31.698541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 670/1331] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:31.703552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.732149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842ad8-6827-4acd-8a3d-16d22b820783 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.747277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.747933 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.750875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.750875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.751075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a42e0e-83e5-4f90-bf4c-6ac2cfad0c4c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.751220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.751220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.767468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.768415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.769481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.769910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.770797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.772732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.805949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 662/1332] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:31.807139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.833633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef99e9b-02c2-4093-bd96-77e038be5474 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.847721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.848658 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.850440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.850928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.851924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.858771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.890439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 663/1333] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:31.891873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.899529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:31.904320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7be58-b304-4ad9-98ac-cc77ae774956 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.911521 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 671/1334] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:31 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:10:31.916161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:31.917013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fed94-0422-4689-9ac2-b5419ac96047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.918690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fed94-0422-4689-9ac2-b5419ac96047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.919366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fed94-0422-4689-9ac2-b5419ac96047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.920010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fed94-0422-4689-9ac2-b5419ac96047 admin admin] PATH_INFO: `/v3/projects/88da27d1fea046dd8a16cb8c09705f23/users/7abccd8676784c218e138db941ce0ce9/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.957399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:31.962070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fed94-0422-4689-9ac2-b5419ac96047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88da27d1fea046dd8a16cb8c09705f23, user_id=7abccd8676784c218e138db941ce0ce9, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.968654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fed94-0422-4689-9ac2-b5419ac96047 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.091225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 664/1335] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:31 2022] PUT /identity/v3/projects/88da27d1fea046dd8a16cb8c09705f23/users/7abccd8676784c218e138db941ce0ce9/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:32.092906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cc214-ed27-40e1-b813-b7438a832946 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.094747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cc214-ed27-40e1-b813-b7438a832946 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.095021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cc214-ed27-40e1-b813-b7438a832946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.132847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b9e3-bdce-42f7-a196-a833bb31417b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.160543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3c5cc214-ed27-40e1-b813-b7438a832946 None None] MFA Rules not processed for user `7abccd8676784c218e138db941ce0ce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:32.163827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3c5cc214-ed27-40e1-b813-b7438a832946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:32.332537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.332537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.332537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.332537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.333884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:32.333884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.365657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 665/1336] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:32.366630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5cc214-ed27-40e1-b813-b7438a832946 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:32.389380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5cc214-ed27-40e1-b813-b7438a832946 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.395083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.396130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.396797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.396797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.397306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:32.399291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.452363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 672/1337] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:31 2022] POST /identity/v3/users => generated 294 bytes in 537 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:10:32.456199 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:32.474874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a1a38-bfd3-4ce5-a419-27e34bb5f003 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.487561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.488696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.488983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.489256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.489908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:32.492019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.523644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 673/1338] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:10:32.527478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:32.554674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9132c7-e77d-46d3-b6c5-bcf00132cd0f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.556579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 666/1339] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:32 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:32.557791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:32.690079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509018-a79e-46b2-a26c-7b02d1ccdc21 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.707867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.707867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.707867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.707867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.708309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:32.710332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.767532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.768477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.768698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.768969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.769629 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:32.788038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.807343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 674/1340] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:10:32.808417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:32.825264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 667/1341] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:32 2022] POST /identity/v3/users => generated 299 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:32.835059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:32.836944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f670f-3a4e-451c-882e-14689df91855 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.847610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ada5-8550-4b0b-98e8-55f123a61233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.848754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ada5-8550-4b0b-98e8-55f123a61233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.849187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ada5-8550-4b0b-98e8-55f123a61233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.850224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ada5-8550-4b0b-98e8-55f123a61233 admin admin] PATH_INFO: `/v3/projects/13d81da9cc664cdf9906020bd7214a9d/users/4d9c293b972c4495ad22f02acb8df8b1/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.857250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b39a8-ba79-4747-958a-cf79dde86d32 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.869752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.870467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.871448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.871927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.872637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:32.874345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.891213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5ada5-8550-4b0b-98e8-55f123a61233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d81da9cc664cdf9906020bd7214a9d, user_id=4d9c293b972c4495ad22f02acb8df8b1, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:32.893755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5ada5-8550-4b0b-98e8-55f123a61233 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:32.911918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 668/1342] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:32.913022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:32.931698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab76c4-d28d-4fd6-be08-e17749757778 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:32.950199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:32.950913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:32.951210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:32.951488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] PATH_INFO: `/v3/projects/56bbf57a891249b3bd90aa6d651e7ac7/users/d5d567e267574b039d8ce6ebf574695e/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:32.986263 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 675/1343] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:32 2022] PUT /identity/v3/projects/13d81da9cc664cdf9906020bd7214a9d/users/4d9c293b972c4495ad22f02acb8df8b1/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:33.000601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a0bd0-d7ae-4fa1-b8e8-44490c46b2ab None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.000942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a0bd0-d7ae-4fa1-b8e8-44490c46b2ab None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.001069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bbf57a891249b3bd90aa6d651e7ac7, user_id=d5d567e267574b039d8ce6ebf574695e, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.001270 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a0bd0-d7ae-4fa1-b8e8-44490c46b2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.003989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:33.113483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a05a0bd0-d7ae-4fa1-b8e8-44490c46b2ab None None] MFA Rules not processed for user `4d9c293b972c4495ad22f02acb8df8b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:33.120641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a05a0bd0-d7ae-4fa1-b8e8-44490c46b2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:33.129582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 669/1344] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:32 2022] PUT /identity/v3/projects/56bbf57a891249b3bd90aa6d651e7ac7/users/d5d567e267574b039d8ce6ebf574695e/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:33.132297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:33.146502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:33.302435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54109b-02ce-4f1c-9708-33f34c62e82c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:33.315466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:33.316143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.316650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.316931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.317591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.319337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:33.330444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 676/1345] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:33.333781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a0bd0-d7ae-4fa1-b8e8-44490c46b2ab None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:33.340472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 670/1346] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:33.345457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:33.362772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a0bd0-d7ae-4fa1-b8e8-44490c46b2ab None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:33.374738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:33.375349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c391ec6-2246-4ef7-824d-455d811631fb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:33.375470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.376011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.376389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.377229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.383101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:33.392506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fc67d-4af6-45f2-95c0-3e66d7d44057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:33.392506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fc67d-4af6-45f2-95c0-3e66d7d44057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.392506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fc67d-4af6-45f2-95c0-3e66d7d44057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.392506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fc67d-4af6-45f2-95c0-3e66d7d44057 admin admin] PATH_INFO: `/v3/projects/56bbf57a891249b3bd90aa6d651e7ac7/users/d5d567e267574b039d8ce6ebf574695e/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.393917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fc67d-4af6-45f2-95c0-3e66d7d44057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bbf57a891249b3bd90aa6d651e7ac7, user_id=d5d567e267574b039d8ce6ebf574695e, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.396667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fc67d-4af6-45f2-95c0-3e66d7d44057 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:33.423666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 671/1347] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:33 2022] PUT /identity/v3/projects/56bbf57a891249b3bd90aa6d651e7ac7/users/d5d567e267574b039d8ce6ebf574695e/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:33.433829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.436439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.436439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.520833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] MFA Rules not processed for user `d5d567e267574b039d8ce6ebf574695e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:33.524446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:33.524579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 677/1348] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:33 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:33.530192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:33.694324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9992-b711-48a7-8ebe-08366164d908 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:33.696311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 672/1349] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:33.713005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:33.713672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.713944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.714221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.715407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.715834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:33.717593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:33.718865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:33.836743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 678/1350] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:33 2022] POST /identity/v3/users => generated 299 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:10:33.838046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:33.852981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22c89d-2679-40ef-b89f-6f837a533eae None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:33.860092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f4649-965c-4032-9548-c12e649e417a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:33.860609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:33.861251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.861538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.861804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.875075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:33.876138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.876138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.876411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.877076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.879002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:33.897054 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.898511 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:33.906380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 679/1351] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:33.907252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:33.918843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 673/1352] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:33.936187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47a433-2326-483b-b12e-fba5b6ce5b35 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:33.942453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:33.943388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:33.943770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:33.944163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] PATH_INFO: `/v3/projects/aeea4240298a4c7694564cc300b315a4/users/35b71ab94edb4630ad210dd66758e670/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:33.992067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeea4240298a4c7694564cc300b315a4, user_id=35b71ab94edb4630ad210dd66758e670, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:33.995234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:34.005211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:34.020415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815a2f6-dcd0-4e62-a01f-dd99cb6e8ff5 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:34.022314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:34.053970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:34.054945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:34.055224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:34.055499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] PATH_INFO: `/v3/users/7abccd8676784c218e138db941ce0ce9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:34.059778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7abccd8676784c218e138db941ce0ce9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:34.061815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:34.093353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 680/1353] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:33 2022] PUT /identity/v3/projects/aeea4240298a4c7694564cc300b315a4/users/35b71ab94edb4630ad210dd66758e670/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:34.100225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:34.196013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd0f49-1e5a-4c32-ac6c-fe7d240bf353 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:34.203672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:34.205578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:34.206469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:34.206846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:34.207693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:34.209724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:34.227382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 681/1354] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:10:34.232591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:34.249946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4d503-644b-48d6-824f-8f81301bb4a8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:34.257248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c3ece-8dd1-4069-8c29-b9c64ef0b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:34.257938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c3ece-8dd1-4069-8c29-b9c64ef0b60a admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:34.258223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c3ece-8dd1-4069-8c29-b9c64ef0b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:34.258501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c3ece-8dd1-4069-8c29-b9c64ef0b60a admin admin] PATH_INFO: `/v3/projects/aeea4240298a4c7694564cc300b315a4/users/35b71ab94edb4630ad210dd66758e670/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:34.260707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c3ece-8dd1-4069-8c29-b9c64ef0b60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeea4240298a4c7694564cc300b315a4, user_id=35b71ab94edb4630ad210dd66758e670, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:34.264863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c3ece-8dd1-4069-8c29-b9c64ef0b60a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:34.293345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 682/1355] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:34 2022] PUT /identity/v3/projects/aeea4240298a4c7694564cc300b315a4/users/35b71ab94edb4630ad210dd66758e670/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:34.293345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:34.293345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:34.293345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:34.327349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] MFA Rules not processed for user `35b71ab94edb4630ad210dd66758e670`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:34.329234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:34.389280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7abccd8676784c218e138db941ce0ce9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:34.410615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7abccd8676784c218e138db941ce0ce9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:34.413705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7abccd8676784c218e138db941ce0ce9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:34.427939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7abccd8676784c218e138db941ce0ce9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:34.432903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 683/1356] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:34.438975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4568aedf-db43-4166-97e4-e8f45f76ed61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7abccd8676784c218e138db941ce0ce9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:34.443115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:34.447320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:34.555482 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 674/1357] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:34 2022] DELETE /identity/v3/users/7abccd8676784c218e138db941ce0ce9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:34.560388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:34.560883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:34.561315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:34.593154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef12692-52f1-4a6b-87c6-401e995752be None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:34.604609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:34.604609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:34.687260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:34.714165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:34.715661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:34.716204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:34.717195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:34.726156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:34.789925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 675/1358] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:34.856572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:34.863597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:34.952904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:34.956472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:35.002477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 684/1359] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:35.016626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef7950-6644-4c77-a2d1-96619771ce45 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:35.024832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576be9f7-d01c-4567-9111-3c39e442d263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:35.025817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be9f7-d01c-4567-9111-3c39e442d263 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:35.026841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be9f7-d01c-4567-9111-3c39e442d263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:35.027343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be9f7-d01c-4567-9111-3c39e442d263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:35.054709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576be9f7-d01c-4567-9111-3c39e442d263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:35.066948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576be9f7-d01c-4567-9111-3c39e442d263 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:35.073611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 676/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:35.133194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:35.137625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:35.168989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9716854-553b-4b83-afa7-75a6f13b08cb service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:35.182458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:35.184000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:35.186840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:35.187936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] PATH_INFO: `/v3/users/4d9c293b972c4495ad22f02acb8df8b1` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:35.191248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9c293b972c4495ad22f02acb8df8b1)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:35.196572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:35.494510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9c293b972c4495ad22f02acb8df8b1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:35.496886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9c293b972c4495ad22f02acb8df8b1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:35.587942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9c293b972c4495ad22f02acb8df8b1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:35.588643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9c293b972c4495ad22f02acb8df8b1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:35.606769 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9c293b972c4495ad22f02acb8df8b1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:35.646517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 685/1361] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:35 2022] DELETE /identity/v3/users/4d9c293b972c4495ad22f02acb8df8b1 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:35.653728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:35.653728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:35.653728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:35.691807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:35.693114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:35.736661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:35.886098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:35.888599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:35.891686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 677/1362] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:35.920709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20451006-b761-4b98-b70f-defebc8741b3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:35.933571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:35.934268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:35.934555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:35.934833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] PATH_INFO: `/v3/users/81b22359bd1c472090b8d12c6f192102` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:35.939992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b22359bd1c472090b8d12c6f192102)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:35.944132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:35.945598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:35.957593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:36.067698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b547c-8e7d-4807-a54e-244dd5230d65 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:36.082727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed18a-9bdc-472e-8848-eca9c8f1cf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:36.083913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed18a-9bdc-472e-8848-eca9c8f1cf80 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:36.084429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed18a-9bdc-472e-8848-eca9c8f1cf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:36.085079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed18a-9bdc-472e-8848-eca9c8f1cf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:36.109858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed18a-9bdc-472e-8848-eca9c8f1cf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:36.114384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed18a-9bdc-472e-8848-eca9c8f1cf80 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:36.130186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 678/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:10:36.197863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b22359bd1c472090b8d12c6f192102'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:36.203608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b22359bd1c472090b8d12c6f192102'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:36.228376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:36.284436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b22359bd1c472090b8d12c6f192102'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:36.285168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b22359bd1c472090b8d12c6f192102'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:36.302781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b22359bd1c472090b8d12c6f192102'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:36.359865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 686/1364] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:35 2022] DELETE /identity/v3/users/81b22359bd1c472090b8d12c6f192102 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:36.365120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a954f-bb71-463f-abcd-878ec2a16323 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:36.370017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a954f-bb71-463f-abcd-878ec2a16323 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:36.370017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a954f-bb71-463f-abcd-878ec2a16323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:36.414505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-253a954f-bb71-463f-abcd-878ec2a16323 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:36.415746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-253a954f-bb71-463f-abcd-878ec2a16323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:36.566273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 679/1365] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:36.685084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:36.686483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:36.693602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:36.693885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:36.694155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:36.753944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] MFA Rules not processed for user `f4c15ef6162348f9b1801c15b959b733`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:36.755186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:36.829524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d062942-2bcc-4e52-97cc-dca94534cb27 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:36.836284 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:36.837272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:36.837760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:36.838241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:36.858299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:36.863234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:36.886784 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 687/1366] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:36.964985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 680/1367] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:36.983776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:36.985472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:37.050899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:37.174413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07fec-c9f2-4baf-833c-02ade259e6da service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:37.189306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:37.189730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:37.190139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:37.190809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:37.213238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:37.215154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:37.235983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 688/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:38.117517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:38.125628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:38.163246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9411-2eb6-456f-9d7a-bfb71ee9810b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:38.165376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:38.168184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:38.193314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:38.194098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:38.194475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:38.195059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] PATH_INFO: `/v3/projects/13d81da9cc664cdf9906020bd7214a9d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:38.201526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d81da9cc664cdf9906020bd7214a9d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:38.206903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9857fe8-4160-4735-b258-4d46880f9989 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:38.211670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:38.220576 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:38.221485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:38.221959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:38.222397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] PATH_INFO: `/v3/projects/88da27d1fea046dd8a16cb8c09705f23` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:38.223833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88da27d1fea046dd8a16cb8c09705f23)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:38.225951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:38.289837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:38.959028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d81da9cc664cdf9906020bd7214a9d', 'request_id': 'req-7707c205-fc16-4a4b-b452-b99fd1b4e255'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.959028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d81da9cc664cdf9906020bd7214a9d', 'request_id': 'req-7707c205-fc16-4a4b-b452-b99fd1b4e255'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.970750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88da27d1fea046dd8a16cb8c09705f23', 'request_id': 'req-59a86c85-c6a1-4d69-a19b-8805107b5697'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.973994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d81da9cc664cdf9906020bd7214a9d', 'request_id': 'req-7707c205-fc16-4a4b-b452-b99fd1b4e255'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.977483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88da27d1fea046dd8a16cb8c09705f23', 'request_id': 'req-59a86c85-c6a1-4d69-a19b-8805107b5697'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.977483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88da27d1fea046dd8a16cb8c09705f23', 'request_id': 'req-59a86c85-c6a1-4d69-a19b-8805107b5697'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.978828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] The token cache is being invalidate because project 13d81da9cc664cdf9906020bd7214a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:38.978828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.978828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7707c205-fc16-4a4b-b452-b99fd1b4e255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:38.978828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 681/1369] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:38 2022] DELETE /identity/v3/projects/13d81da9cc664cdf9906020bd7214a9d => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:38.978828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:38.980955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:38.980955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:39.002419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:39.059096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] MFA Rules not processed for user `83e3e4b159c94fdf9a03b72ebf62ac46`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:39.061044 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:39.146780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] The token cache is being invalidate because project 88da27d1fea046dd8a16cb8c09705f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:39.147588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.153553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.158408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 689/1370] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:38 2022] DELETE /identity/v3/projects/88da27d1fea046dd8a16cb8c09705f23 => generated 0 bytes in 991 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:39.159516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:39.161445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:39.248886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 682/1371] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:39.250452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:39.255839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:39.359187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a86c85-c6a1-4d69-a19b-8805107b5697 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:39.372650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:39.373401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:39.374622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:39.375264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] PATH_INFO: `/v3/projects/6b165f81d6714550af6432b58e3af800` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:39.381978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b165f81d6714550af6432b58e3af800)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:39.387310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:39.456660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c69f04-af9d-424e-995a-3b6304c788a5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:39.462383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:39.463157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:39.463560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:39.464131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] PATH_INFO: `/v3/users/35b71ab94edb4630ad210dd66758e670` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:39.466375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b71ab94edb4630ad210dd66758e670)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:39.469858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:39.624036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b165f81d6714550af6432b58e3af800', 'request_id': 'req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.652825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b165f81d6714550af6432b58e3af800', 'request_id': 'req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.670880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b165f81d6714550af6432b58e3af800', 'request_id': 'req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.702467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:39.773321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b71ab94edb4630ad210dd66758e670'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.795744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] The token cache is being invalidate because project 6b165f81d6714550af6432b58e3af800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:39.795744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.796858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-65f77372-b03c-4ee7-bbac-c3afbfdf5d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.799054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 690/1372] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:39 2022] DELETE /identity/v3/projects/6b165f81d6714550af6432b58e3af800 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:39.800620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a637-9f07-4748-9e31-5e420c387fb7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:39.801045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a637-9f07-4748-9e31-5e420c387fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:39.801436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a637-9f07-4748-9e31-5e420c387fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:39.809900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b71ab94edb4630ad210dd66758e670'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.810778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b71ab94edb4630ad210dd66758e670'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.826520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b71ab94edb4630ad210dd66758e670'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.827717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b71ab94edb4630ad210dd66758e670'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:39.872523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1942a637-9f07-4748-9e31-5e420c387fb7 None None] MFA Rules not processed for user `95eef108b14a4a90a8b617088c904a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:39.874060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1942a637-9f07-4748-9e31-5e420c387fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.005051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 683/1373] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:39 2022] DELETE /identity/v3/users/35b71ab94edb4630ad210dd66758e670 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:40.011451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:40.016382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.021840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 691/1374] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:40.023506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fcb83-16c9-4e9e-b528-682f85a1a287 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.024110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fcb83-16c9-4e9e-b528-682f85a1a287 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.026252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fcb83-16c9-4e9e-b528-682f85a1a287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.062829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-298fcb83-16c9-4e9e-b528-682f85a1a287 None None] MFA Rules not processed for user `83e3e4b159c94fdf9a03b72ebf62ac46`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:40.064338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-298fcb83-16c9-4e9e-b528-682f85a1a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.176361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 692/1375] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:40.176361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.176361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.176361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.181006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24109db7-fe09-4019-9f33-c332497beec2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:40.199926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:40.200448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:40.201113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.201396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.201671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] PATH_INFO: `/v3/users/d5d567e267574b039d8ce6ebf574695e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.202676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.205103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d567e267574b039d8ce6ebf574695e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:40.207075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:40.221858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 693/1376] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:40.223341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:40.225934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.345038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7069b-a726-4277-8cce-2def2adcda7d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:40.357585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:40.359254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.360192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.360682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.441335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d567e267574b039d8ce6ebf574695e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:40.465893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:40.469137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:40.473020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d567e267574b039d8ce6ebf574695e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:40.477210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d567e267574b039d8ce6ebf574695e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:40.491121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 694/1377] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:40.492536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:40.493425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d567e267574b039d8ce6ebf574695e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:40.494724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-25c2ece7-a854-4b3c-a4c5-c22df5a13eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d567e267574b039d8ce6ebf574695e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:40.500167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.529477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92626b72-3847-4fa7-894f-9ff6df19a4ef service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:40.544213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:40.545326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.546419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.546792 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.549206 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.584185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 684/1378] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:40 2022] DELETE /identity/v3/users/d5d567e267574b039d8ce6ebf574695e => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:40.584185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b23602-7db8-4923-afd1-cc195ae5fef1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.584185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b23602-7db8-4923-afd1-cc195ae5fef1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.584185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b23602-7db8-4923-afd1-cc195ae5fef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.584577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:40.584577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:40.611042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 695/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:40.616625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-65b23602-7db8-4923-afd1-cc195ae5fef1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:40.618223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-65b23602-7db8-4923-afd1-cc195ae5fef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:40.769762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 685/1380] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:40.773727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:40.794182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f786ad7-dbca-481c-a84d-a68049659077 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:40.808212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:40.808961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.809270 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.813417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.813417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:40.813417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:40.813417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:40.840886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 696/1381] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:40.848011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b23602-7db8-4923-afd1-cc195ae5fef1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:40.856154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b23602-7db8-4923-afd1-cc195ae5fef1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:40.861600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:40.862407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:40.862692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:40.862960 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:40.863611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:40.866377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.004778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 686/1382] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:40 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:10:41.008950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:41.022748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fbb72-4246-47ab-948a-321bbdf7d640 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:41.154425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:41.156653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:41.160102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecf3c1-677d-4a1c-8f60-052bd8fd066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:41.160880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecf3c1-677d-4a1c-8f60-052bd8fd066b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.161223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecf3c1-677d-4a1c-8f60-052bd8fd066b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.161578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecf3c1-677d-4a1c-8f60-052bd8fd066b admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.162287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecf3c1-677d-4a1c-8f60-052bd8fd066b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:41.165323 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecf3c1-677d-4a1c-8f60-052bd8fd066b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.187324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321ed71-e3cc-4b4a-8f46-dfcfa18494dc admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:41.209812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:41.210692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.211133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.211540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] PATH_INFO: `/v3/projects/aeea4240298a4c7694564cc300b315a4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.215760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeea4240298a4c7694564cc300b315a4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:41.218725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.320522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 697/1383] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:41 2022] POST /identity/v3/users => generated 285 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:41.322085 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b04f76-ae9d-4fca-932b-78c665493a99 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.322578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b04f76-ae9d-4fca-932b-78c665493a99 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.323051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b04f76-ae9d-4fca-932b-78c665493a99 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.325167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 698/1384] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 26 11:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:41.332942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b04f76-ae9d-4fca-932b-78c665493a99 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:41.352076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b04f76-ae9d-4fca-932b-78c665493a99 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:41.360035 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59922b-686e-4a8c-8d7f-cde051c1b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:41.361286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59922b-686e-4a8c-8d7f-cde051c1b5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.361601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59922b-686e-4a8c-8d7f-cde051c1b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.361838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59922b-686e-4a8c-8d7f-cde051c1b5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.362836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59922b-686e-4a8c-8d7f-cde051c1b5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:41.364208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59922b-686e-4a8c-8d7f-cde051c1b5b1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.397734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 699/1385] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:41.399329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563bc1a-4079-426e-8f02-b32f45851942 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.399617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563bc1a-4079-426e-8f02-b32f45851942 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.399924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563bc1a-4079-426e-8f02-b32f45851942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c563bc1a-4079-426e-8f02-b32f45851942 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:10:41.419517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:41.420285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:10:41.431019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeea4240298a4c7694564cc300b315a4', 'request_id': 'req-8dae2e29-9657-4c76-986c-4d98d00f80bf'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:41.437240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 700/1386] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:41 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 39 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:10:41.439097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeea4240298a4c7694564cc300b315a4', 'request_id': 'req-8dae2e29-9657-4c76-986c-4d98d00f80bf'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:41.439584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563bc1a-4079-426e-8f02-b32f45851942 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:41.454056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeea4240298a4c7694564cc300b315a4', 'request_id': 'req-8dae2e29-9657-4c76-986c-4d98d00f80bf'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:41.560588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] The token cache is being invalidate because project aeea4240298a4c7694564cc300b315a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:41.565923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:41.565923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-8dae2e29-9657-4c76-986c-4d98d00f80bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:41.566800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 687/1387] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:41 2022] DELETE /identity/v3/projects/aeea4240298a4c7694564cc300b315a4 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:41.568302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c9d3e-5239-4e9f-9d72-f891c60fc370 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.570879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c9d3e-5239-4e9f-9d72-f891c60fc370 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.571197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c9d3e-5239-4e9f-9d72-f891c60fc370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.593332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563bc1a-4079-426e-8f02-b32f45851942 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:41.607628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:41.608686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.609166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.610290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.611173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:41.614352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.620143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3a8c9d3e-5239-4e9f-9d72-f891c60fc370 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:41.625208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3a8c9d3e-5239-4e9f-9d72-f891c60fc370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:41.646305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 701/1388] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:41.649311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:41.652879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 688/1389] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:41.653588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:41.654618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb9b5-e785-4f83-a49f-80a7e8319cf1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.654902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb9b5-e785-4f83-a49f-80a7e8319cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.658842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb9b5-e785-4f83-a49f-80a7e8319cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.672666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0932146-60dc-4310-8c14-64efbfc6b5e9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:41.693095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc2df5-7a3d-433d-9b1a-7605c039821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:41.693999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2df5-7a3d-433d-9b1a-7605c039821e admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.694280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2df5-7a3d-433d-9b1a-7605c039821e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.694568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc2df5-7a3d-433d-9b1a-7605c039821e admin admin] PATH_INFO: `/v3/projects/277708e8fc534cc4bfb5af23735367da/users/eed05c0d011f43449e73c384513e03df/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.704893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f2bfb9b5-e785-4f83-a49f-80a7e8319cf1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:41.709101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f2bfb9b5-e785-4f83-a49f-80a7e8319cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:41.726883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 689/1390] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:41.733736 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfb9b5-e785-4f83-a49f-80a7e8319cf1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:41.746927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc2df5-7a3d-433d-9b1a-7605c039821e admin admin] RBAC: Authorizing `identity:create_grant(project_id=277708e8fc534cc4bfb5af23735367da, user_id=eed05c0d011f43449e73c384513e03df, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:41.749682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc2df5-7a3d-433d-9b1a-7605c039821e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.753917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfb9b5-e785-4f83-a49f-80a7e8319cf1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:41.768843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:41.772135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.772135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.772135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.773420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:41.774164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:41.778224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.794756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:41.808258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 690/1391] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:41.809246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:41.831047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cca78-2d9f-47db-8c38-ad9ecb5b1d4c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:41.842881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:41.843515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.846634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.846753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.847434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:41.861779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 702/1392] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:41 2022] PUT /identity/v3/projects/277708e8fc534cc4bfb5af23735367da/users/eed05c0d011f43449e73c384513e03df/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:41.861998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:41.863601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84038663-5b30-4231-875b-1dfedefd2ad6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.863898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84038663-5b30-4231-875b-1dfedefd2ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.864174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84038663-5b30-4231-875b-1dfedefd2ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.926460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-84038663-5b30-4231-875b-1dfedefd2ad6 None None] MFA Rules not processed for user `eed05c0d011f43449e73c384513e03df`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:41.927718 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-84038663-5b30-4231-875b-1dfedefd2ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:42.013970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 691/1393] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:41 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:42.020386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:42.136904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 703/1394] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:42.138619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84038663-5b30-4231-875b-1dfedefd2ad6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:42.182345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:42.188466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57edb-60e7-439a-9005-2861a61080d3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:42.202298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:42.211822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:42.212220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:42.212542 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:42.213244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:42.217498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:42.346804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 692/1395] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:42 2022] POST /identity/v3/users => generated 291 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:42.351725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:42.357992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:42.394674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f5065-c9cd-438c-b325-4291b5bd4e52 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:42.403725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84038663-5b30-4231-875b-1dfedefd2ad6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:42.412459 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:42.413195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:42.413583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:42.413964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] PATH_INFO: `/v3/projects/56bbf57a891249b3bd90aa6d651e7ac7` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:42.415547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] RBAC: Authorizing `identity:delete_project(project_id=56bbf57a891249b3bd90aa6d651e7ac7)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:42.418249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:42.424038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:42.425246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:42.425826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:42.428269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:42.428895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:42.434783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:42.617153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56bbf57a891249b3bd90aa6d651e7ac7', 'request_id': 'req-5bc6161a-a117-42a8-bf11-e2304d96818f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:42.620694 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 704/1396] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:42 2022] POST /identity/v3/projects => generated 367 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:42.624954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:42.646342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56bbf57a891249b3bd90aa6d651e7ac7', 'request_id': 'req-5bc6161a-a117-42a8-bf11-e2304d96818f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:42.673216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56bbf57a891249b3bd90aa6d651e7ac7', 'request_id': 'req-5bc6161a-a117-42a8-bf11-e2304d96818f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:42.796930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] The token cache is being invalidate because project 56bbf57a891249b3bd90aa6d651e7ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:42.797371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:42.798831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:42.804255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 693/1397] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:42 2022] DELETE /identity/v3/projects/56bbf57a891249b3bd90aa6d651e7ac7 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:42.804930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:42.814603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:42.882643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e389-8619-4d82-9bdd-ac94dfcecfac admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:42.896115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a9f60-9ba1-4737-8674-786f188e5ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:42.896767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a9f60-9ba1-4737-8674-786f188e5ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:42.897054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a9f60-9ba1-4737-8674-786f188e5ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:42.897331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a9f60-9ba1-4737-8674-786f188e5ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:42.897989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a9f60-9ba1-4737-8674-786f188e5ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:42.902752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a9f60-9ba1-4737-8674-786f188e5ff9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:42.936511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 705/1398] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:42.939308 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae044b5e-455b-4750-8fea-8e0433863f2d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:42.939433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae044b5e-455b-4750-8fea-8e0433863f2d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:42.940415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae044b5e-455b-4750-8fea-8e0433863f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:42.992371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6161a-a117-42a8-bf11-e2304d96818f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:43.004912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a03e4-968f-40da-a919-f4a0ec5a38ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:43.007758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a03e4-968f-40da-a919-f4a0ec5a38ac admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.010655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a03e4-968f-40da-a919-f4a0ec5a38ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.011769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a03e4-968f-40da-a919-f4a0ec5a38ac admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.016546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a03e4-968f-40da-a919-f4a0ec5a38ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:43.028081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a03e4-968f-40da-a919-f4a0ec5a38ac admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:43.044996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ae044b5e-455b-4750-8fea-8e0433863f2d None None] MFA Rules not processed for user `bfa99ee8ae73487da15a4adf72c473cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:43.047197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ae044b5e-455b-4750-8fea-8e0433863f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.159410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 694/1399] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:42 2022] POST /identity/v3/users => generated 290 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:43.163732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f180d2-9604-439f-9311-6a7bf9e220eb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.164101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f180d2-9604-439f-9311-6a7bf9e220eb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.164383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f180d2-9604-439f-9311-6a7bf9e220eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.201627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 706/1400] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:43.202988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.203311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.203584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.226105 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:43.227624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.238327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 707/1401] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:43.239476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:43.240969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.255825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580f8d7-b5ad-4a2c-ad12-32153d8c54a9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:43.261972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:43.262787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.263252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.263664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.264508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:43.266319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:43.280356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 708/1402] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:43.281531 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:43.300483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-33f180d2-9604-439f-9311-6a7bf9e220eb None None] MFA Rules not processed for user `eca696f5dfaa466fa06782c82ea42758`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:43.303747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431c356-878f-47b5-a95c-f8bc8f42e3a0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:43.305780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-33f180d2-9604-439f-9311-6a7bf9e220eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.310948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af805a-04f6-40b8-8812-ac4ea7bc78b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:43.312408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af805a-04f6-40b8-8812-ac4ea7bc78b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.312835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af805a-04f6-40b8-8812-ac4ea7bc78b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.313288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af805a-04f6-40b8-8812-ac4ea7bc78b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.314183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af805a-04f6-40b8-8812-ac4ea7bc78b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:43.316514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af805a-04f6-40b8-8812-ac4ea7bc78b3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:43.340343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 709/1403] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:43.347300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0f5e-656f-4807-8ad3-ca2f65b3b6cb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.347781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0f5e-656f-4807-8ad3-ca2f65b3b6cb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.349255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0f5e-656f-4807-8ad3-ca2f65b3b6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.415554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-850c0f5e-656f-4807-8ad3-ca2f65b3b6cb None None] MFA Rules not processed for user `bfa99ee8ae73487da15a4adf72c473cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:43.415554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-850c0f5e-656f-4807-8ad3-ca2f65b3b6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.437234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 710/1404] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:43.437234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429b7ba-dc26-46e7-850a-8001f5a6e77e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.437234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429b7ba-dc26-46e7-850a-8001f5a6e77e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.437234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429b7ba-dc26-46e7-850a-8001f5a6e77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.486907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b429b7ba-dc26-46e7-850a-8001f5a6e77e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:43.488639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b429b7ba-dc26-46e7-850a-8001f5a6e77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.511934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 711/1405] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:43.513712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429b7ba-dc26-46e7-850a-8001f5a6e77e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:43.518397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 695/1406] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:43.520377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f180d2-9604-439f-9311-6a7bf9e220eb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:43.546800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429b7ba-dc26-46e7-850a-8001f5a6e77e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:43.558665 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f180d2-9604-439f-9311-6a7bf9e220eb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:43.562192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:43.563305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.563825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.564170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] PATH_INFO: `/v3/projects/f4ce7ac3000b42ac980d2c1c04a7bd95/users/24da5700a7554b59af438bbc62eb393a/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.570390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:43.570952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.572209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.573629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] PATH_INFO: `/v3/projects/2d166711262f46df919bd33fa02d0671/users/fd42749682e3439a89b58113bf6e2b54/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.597159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ce7ac3000b42ac980d2c1c04a7bd95, user_id=24da5700a7554b59af438bbc62eb393a, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:43.600058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:43.606425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d166711262f46df919bd33fa02d0671, user_id=fd42749682e3439a89b58113bf6e2b54, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:43.611164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:43.640015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:43.749665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 712/1407] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:43 2022] PUT /identity/v3/projects/f4ce7ac3000b42ac980d2c1c04a7bd95/users/24da5700a7554b59af438bbc62eb393a/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:43.760033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 696/1408] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:43 2022] PUT /identity/v3/projects/2d166711262f46df919bd33fa02d0671/users/fd42749682e3439a89b58113bf6e2b54/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:43.761401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:43.761485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:43.765293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.874746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc8364-fc66-45fe-b56c-426924905335 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:43.890750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:43.894695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:43.894840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:43.895014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:43.896512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:43.897160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:43.900429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:43.929829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 697/1409] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:43.930885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:43.934735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:43.942605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2459b-92b5-48db-a879-e1ac6ba80872 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:43.949045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:44.134183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93906c-4b73-488f-96ab-1edba1332f30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:44.135061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93906c-4b73-488f-96ab-1edba1332f30 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.138648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93906c-4b73-488f-96ab-1edba1332f30 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.138790 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93906c-4b73-488f-96ab-1edba1332f30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.197435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42877cd-f7fa-4cee-b882-55b8873d8f1b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:44.215656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:44.216580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.217053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.217527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.294199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:44.296872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:44.313419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 698/1410] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:44.314676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:44.315551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93906c-4b73-488f-96ab-1edba1332f30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:44.318295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93906c-4b73-488f-96ab-1edba1332f30 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:44.329812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58376c7-565b-4bf8-acb3-7792e847d6bd service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:44.342695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:44.343667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.344228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.344644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.347089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:44.350294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:44.350969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 713/1411] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:44.352896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ab2a-edce-4c0e-80bf-f18737ae4b98 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.353244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ab2a-edce-4c0e-80bf-f18737ae4b98 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.353661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ab2a-edce-4c0e-80bf-f18737ae4b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.393907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d257ab2a-edce-4c0e-80bf-f18737ae4b98 None None] MFA Rules not processed for user `24da5700a7554b59af438bbc62eb393a`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:44.395780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d257ab2a-edce-4c0e-80bf-f18737ae4b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:44.396925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 699/1412] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:44.399277 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:44.421582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb33f9-052c-4adc-862e-299ec005b34d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:44.437210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ea6d1-0141-4904-af5a-377c65e8071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:44.439337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ea6d1-0141-4904-af5a-377c65e8071b admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.439868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ea6d1-0141-4904-af5a-377c65e8071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.446281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ea6d1-0141-4904-af5a-377c65e8071b admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.447154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ea6d1-0141-4904-af5a-377c65e8071b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:44.451171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ea6d1-0141-4904-af5a-377c65e8071b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:44.488966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 714/1413] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:44.488966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ab2a-edce-4c0e-80bf-f18737ae4b98 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:44.536111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ab2a-edce-4c0e-80bf-f18737ae4b98 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:44.541280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:44.549163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.552122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.552555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] PATH_INFO: `/v3/projects/2d166711262f46df919bd33fa02d0671/users/fd42749682e3439a89b58113bf6e2b54/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.558376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d166711262f46df919bd33fa02d0671, user_id=fd42749682e3439a89b58113bf6e2b54, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:44.562953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:44.577786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 715/1414] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:44 2022] PUT /identity/v3/projects/2d166711262f46df919bd33fa02d0671/users/fd42749682e3439a89b58113bf6e2b54/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:44.578902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:44.601784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 700/1415] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:44 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:44.603003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff421-f5cf-4f9c-b243-b1910c6cc500 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.603213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff421-f5cf-4f9c-b243-b1910c6cc500 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.603487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff421-f5cf-4f9c-b243-b1910c6cc500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.656477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-025ff421-f5cf-4f9c-b243-b1910c6cc500 None None] MFA Rules not processed for user `fd42749682e3439a89b58113bf6e2b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:44.659084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-025ff421-f5cf-4f9c-b243-b1910c6cc500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:44.666311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e601ff-bb4e-4b45-8df5-d92b81210915 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:44.753591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:44.831548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:44.834534 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.835094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.840625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.841350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:44.843310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:44.850746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 701/1416] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:44.852435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff421-f5cf-4f9c-b243-b1910c6cc500 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:44.874007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff421-f5cf-4f9c-b243-b1910c6cc500 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:44.881635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:44.882606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:44.883165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:44.883579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:44.884664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:44.887171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.017024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 702/1417] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:44 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:45.017969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:45.022132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:45.024350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 716/1418] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:44 2022] POST /identity/v3/projects => generated 379 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:45.025959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:45.172830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171b86d-c5ca-4cff-beba-2f3dbe05c86c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:45.215880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:45.232245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4080ac1-4296-42e2-bdf9-b6d3918b891a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:45.245854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:45.246498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.246782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.247054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.250091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:45.281691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.281813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6673f9-ea36-4bcf-932a-b3affd2e6172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:45.282417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6673f9-ea36-4bcf-932a-b3affd2e6172 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.282699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6673f9-ea36-4bcf-932a-b3affd2e6172 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.282970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6673f9-ea36-4bcf-932a-b3affd2e6172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.335709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 717/1419] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:45.339678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:45.370990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580e29b-148f-400e-96be-79f847e2a810 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:45.384774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:45.384774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.384774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.385390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.386265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:45.392247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.437408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6673f9-ea36-4bcf-932a-b3affd2e6172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:45.440159 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6673f9-ea36-4bcf-932a-b3affd2e6172 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.444832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:45.458229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 703/1420] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:45.474593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dab2c-ab14-4c22-8f89-bab656d30dc1 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.475048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dab2c-ab14-4c22-8f89-bab656d30dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.475471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dab2c-ab14-4c22-8f89-bab656d30dc1 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.482359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 704/1421] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 26 11:10:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:45.483437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dab2c-ab14-4c22-8f89-bab656d30dc1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:45.533492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dab2c-ab14-4c22-8f89-bab656d30dc1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:45.548765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb01070-72ab-4fe5-8a7e-cb240dbec4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:45.550217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb01070-72ab-4fe5-8a7e-cb240dbec4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.552371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb01070-72ab-4fe5-8a7e-cb240dbec4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.552371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb01070-72ab-4fe5-8a7e-cb240dbec4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.552371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb01070-72ab-4fe5-8a7e-cb240dbec4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:45.554813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb01070-72ab-4fe5-8a7e-cb240dbec4e1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.588265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 705/1422] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:45.591096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96087ce5-caee-46bd-8c67-b481ff14bce0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.591773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96087ce5-caee-46bd-8c67-b481ff14bce0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.593768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96087ce5-caee-46bd-8c67-b481ff14bce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.594712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 718/1423] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:45 2022] POST /identity/v3/users => generated 296 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:45.595541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-96087ce5-caee-46bd-8c67-b481ff14bce0 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:10:45.606842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:45.607679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 11:10:45.609380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 706/1424] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:45 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 23 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:10:45.610663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b13001-d1dd-4163-a02e-83e1bc82608c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.610949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b13001-d1dd-4163-a02e-83e1bc82608c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.611220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b13001-d1dd-4163-a02e-83e1bc82608c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.626492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd05bf-12b2-4557-85b7-9d75537f3082 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:45.635934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f0b13001-d1dd-4163-a02e-83e1bc82608c None None] MFA Rules not processed for user `f4c15ef6162348f9b1801c15b959b733`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:45.636083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6512a0-a178-4ba7-8967-cbf689cc8472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:45.636688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6512a0-a178-4ba7-8967-cbf689cc8472 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.636982 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6512a0-a178-4ba7-8967-cbf689cc8472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.637254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6512a0-a178-4ba7-8967-cbf689cc8472 admin admin] PATH_INFO: `/v3/projects/fc8251e1048c4416a6eb0204c73066ce/users/430a8beb34924e54ad6ba7b8370a110a/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.638753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f0b13001-d1dd-4163-a02e-83e1bc82608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:45.680735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6512a0-a178-4ba7-8967-cbf689cc8472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc8251e1048c4416a6eb0204c73066ce, user_id=430a8beb34924e54ad6ba7b8370a110a, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:45.686264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6512a0-a178-4ba7-8967-cbf689cc8472 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.768473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 707/1425] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:45.769625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b13001-d1dd-4163-a02e-83e1bc82608c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:45.788400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b13001-d1dd-4163-a02e-83e1bc82608c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:45.793523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:45.794314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.794609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.823830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.823830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:45.823830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.823830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 708/1426] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:45.827251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:45.840078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8b4a8-914f-46db-b0da-1ac3c3648c10 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:45.858942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 719/1427] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:45 2022] PUT /identity/v3/projects/fc8251e1048c4416a6eb0204c73066ce/users/430a8beb34924e54ad6ba7b8370a110a/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:45.870727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad22963-1b53-4505-b5f9-940a858cb90f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.872445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad22963-1b53-4505-b5f9-940a858cb90f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.872813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad22963-1b53-4505-b5f9-940a858cb90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.875742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:45.876525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.876809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.877088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] PATH_INFO: `/v3/projects/b06c6d2db8c0414598760d021f139edf/users/1a281c8fdcfe4b0db7a8d1daf24eb5f0/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.899817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06c6d2db8c0414598760d021f139edf, user_id=1a281c8fdcfe4b0db7a8d1daf24eb5f0, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:45.903354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:45.936612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-aad22963-1b53-4505-b5f9-940a858cb90f None None] MFA Rules not processed for user `430a8beb34924e54ad6ba7b8370a110a`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:45.940580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aad22963-1b53-4505-b5f9-940a858cb90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:46.016154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 709/1428] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:45 2022] PUT /identity/v3/projects/b06c6d2db8c0414598760d021f139edf/users/1a281c8fdcfe4b0db7a8d1daf24eb5f0/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:46.020812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:46.080398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 720/1429] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:46.081547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad22963-1b53-4505-b5f9-940a858cb90f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:46.205967 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5bffb-30b1-4e40-96ad-7b4cf9af9baf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:46.219077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:46.222419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:46.222419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:46.222419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:46.222967 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:46.226928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:46.255767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 710/1430] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:10:46.257092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:46.279522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab89bf9-7c04-4b89-9166-bd34f2e4777b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:46.288591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad22963-1b53-4505-b5f9-940a858cb90f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:46.296753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01714768-9c78-4f71-b68f-d593ad229f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:46.299169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:46.300437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:46.300437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:46.300630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01714768-9c78-4f71-b68f-d593ad229f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:46.300733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:46.300817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01714768-9c78-4f71-b68f-d593ad229f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:46.300817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01714768-9c78-4f71-b68f-d593ad229f87 admin admin] PATH_INFO: `/v3/projects/b06c6d2db8c0414598760d021f139edf/users/1a281c8fdcfe4b0db7a8d1daf24eb5f0/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:46.301074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:46.303094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:46.306245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01714768-9c78-4f71-b68f-d593ad229f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06c6d2db8c0414598760d021f139edf, user_id=1a281c8fdcfe4b0db7a8d1daf24eb5f0, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:46.312979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01714768-9c78-4f71-b68f-d593ad229f87 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:46.371746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 711/1431] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:46 2022] PUT /identity/v3/projects/b06c6d2db8c0414598760d021f139edf/users/1a281c8fdcfe4b0db7a8d1daf24eb5f0/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:46.382556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411bb61-27b9-4413-b29f-929ce7e2d9b6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:46.382880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411bb61-27b9-4413-b29f-929ce7e2d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:46.383139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411bb61-27b9-4413-b29f-929ce7e2d9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:46.469302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2411bb61-27b9-4413-b29f-929ce7e2d9b6 None None] MFA Rules not processed for user `1a281c8fdcfe4b0db7a8d1daf24eb5f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:46.470548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2411bb61-27b9-4413-b29f-929ce7e2d9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:46.503321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 721/1432] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:46 2022] POST /identity/v3/projects => generated 367 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:46.508423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:46.723416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 712/1433] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:46.737225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411bb61-27b9-4413-b29f-929ce7e2d9b6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:46.758502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c3b6-7f86-4ab5-be88-6a97feccf47e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:46.779939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff73fa-62fa-43ac-b590-2eaf2238951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:46.780627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff73fa-62fa-43ac-b590-2eaf2238951a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:46.781645 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff73fa-62fa-43ac-b590-2eaf2238951a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:46.781940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff73fa-62fa-43ac-b590-2eaf2238951a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:46.782728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff73fa-62fa-43ac-b590-2eaf2238951a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:46.791201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff73fa-62fa-43ac-b590-2eaf2238951a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:46.893506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411bb61-27b9-4413-b29f-929ce7e2d9b6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:46.940671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 722/1434] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:46 2022] POST /identity/v3/users => generated 290 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:46.942320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4dca6-0786-4510-9713-a8f22fd0f5cb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:46.942616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4dca6-0786-4510-9713-a8f22fd0f5cb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:46.942892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4dca6-0786-4510-9713-a8f22fd0f5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.007009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-06b4dca6-0786-4510-9713-a8f22fd0f5cb None None] MFA Rules not processed for user `95eef108b14a4a90a8b617088c904a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:47.007009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-06b4dca6-0786-4510-9713-a8f22fd0f5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:47.049190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:47.051099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.052906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.054008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.195499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 723/1435] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:47.199881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4dca6-0786-4510-9713-a8f22fd0f5cb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:47.230510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4dca6-0786-4510-9713-a8f22fd0f5cb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:47.246420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:47.247642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.250376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.250876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.256493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:47.269774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:47.281703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:47.283261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:47.302360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 724/1436] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:47.307858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:47.327720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 713/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:10:47.327891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55212a06-2a16-4a0a-93d1-de4ceb595fc5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:47.339809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:47.340770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.341253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.366900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] PATH_INFO: `/v3/projects/ed66b79140564481a906f7b8e5f301a2/users/7995776b6de54f88bf9150dd1a51879c/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.393158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed66b79140564481a906f7b8e5f301a2, user_id=7995776b6de54f88bf9150dd1a51879c, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:47.399887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:47.489726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 725/1438] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:47 2022] PUT /identity/v3/projects/ed66b79140564481a906f7b8e5f301a2/users/7995776b6de54f88bf9150dd1a51879c/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:47.493590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:47.514381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158ec8c-e9da-4e9e-98ef-c3ad94cf072f service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:47.636286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a10cd9-c268-4c42-b141-10ac00d15632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:47.637048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a10cd9-c268-4c42-b141-10ac00d15632 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.637402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a10cd9-c268-4c42-b141-10ac00d15632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.637749 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a10cd9-c268-4c42-b141-10ac00d15632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.638617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a10cd9-c268-4c42-b141-10ac00d15632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:47.644693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a10cd9-c268-4c42-b141-10ac00d15632 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:47.650662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:47.666144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 714/1439] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:47.668996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:47.681625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afb52a-b43b-4f84-b421-8b7969df2074 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:47.690819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa399b-0a01-4aad-9bae-fd87fdadff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:47.691506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa399b-0a01-4aad-9bae-fd87fdadff0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.691809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa399b-0a01-4aad-9bae-fd87fdadff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.692263 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa399b-0a01-4aad-9bae-fd87fdadff0a admin admin] PATH_INFO: `/v3/projects/ed66b79140564481a906f7b8e5f301a2/users/7995776b6de54f88bf9150dd1a51879c/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.697172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa399b-0a01-4aad-9bae-fd87fdadff0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed66b79140564481a906f7b8e5f301a2, user_id=7995776b6de54f88bf9150dd1a51879c, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:47.701421 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa399b-0a01-4aad-9bae-fd87fdadff0a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:47.764004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 726/1440] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:47 2022] PUT /identity/v3/projects/ed66b79140564481a906f7b8e5f301a2/users/7995776b6de54f88bf9150dd1a51879c/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:47.764603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d0886-e25e-4ba6-b4a9-1b11dd1655ee None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.764873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d0886-e25e-4ba6-b4a9-1b11dd1655ee None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.765155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d0886-e25e-4ba6-b4a9-1b11dd1655ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.807303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-973d0886-e25e-4ba6-b4a9-1b11dd1655ee None None] MFA Rules not processed for user `7995776b6de54f88bf9150dd1a51879c`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:47.809061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-973d0886-e25e-4ba6-b4a9-1b11dd1655ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:47.891409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcaa11-4bc5-432d-a83d-3276548615e9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.891409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcaa11-4bc5-432d-a83d-3276548615e9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.891409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcaa11-4bc5-432d-a83d-3276548615e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-89fcaa11-4bc5-432d-a83d-3276548615e9 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:10:47.903708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:47.904580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:10:47.911718 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 727/1441] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:47 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 26 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:10:47.961560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 715/1442] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:48.110861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcaa11-4bc5-432d-a83d-3276548615e9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:48.117086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-89fcaa11-4bc5-432d-a83d-3276548615e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:48.194977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:48.265117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcaa11-4bc5-432d-a83d-3276548615e9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:48.270200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:48.281722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:48.282125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:48.283445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:48.297852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:48.300194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:48.319137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 728/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:48.640153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef76d-d70e-4456-9033-1b0202371e47 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:48.640447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef76d-d70e-4456-9033-1b0202371e47 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:48.640751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef76d-d70e-4456-9033-1b0202371e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-556ef76d-d70e-4456-9033-1b0202371e47 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:10:48.654735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:48.655688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 11:10:48.661638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 716/1444] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:48 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 19 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 26 11:10:48.797418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:48.841231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ef76d-d70e-4456-9033-1b0202371e47 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:48.882725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ef76d-d70e-4456-9033-1b0202371e47 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:48.897305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:48.897948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:48.898560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:48.898843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:48.931816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e552c30-21c6-4045-91d5-13261c219c23 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:48.942766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a281c8fdcfe4b0db7a8d1daf24eb5f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06c6d2db8c0414598760d021f139edf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:48.943750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:48.945141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:48.947486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:48.949708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:48.951344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:48.955536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:48.995377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 729/1445] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:48.996655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:49.017312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd542-baa9-4f11-95ba-e75383e88b62 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:49.030190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a281c8fdcfe4b0db7a8d1daf24eb5f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06c6d2db8c0414598760d021f139edf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:49.033108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:49.033760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:49.035697 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:49.036315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:49.037541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:49.038429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:49.046707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:49.064853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 717/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:49.197334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:49.207165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:49.208530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 730/1447] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:48 2022] POST /identity/v3/projects => generated 354 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:49.215402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dd987-51bd-4b8a-bb9d-2c01277c497c service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:49.357358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a281c8fdcfe4b0db7a8d1daf24eb5f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06c6d2db8c0414598760d021f139edf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:49.359114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:49.361044 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:49.361355 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:49.362201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:49.362521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:49.373090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:49.400906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 718/1448] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:49.433738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:49.476824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:49.493538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e52fc-7768-4273-8a0c-2787f50efc56 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:49.501915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a281c8fdcfe4b0db7a8d1daf24eb5f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b06c6d2db8c0414598760d021f139edf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:49.502833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:49.503257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:49.503385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] PATH_INFO: `/v3/projects/0627b19238764d9b8774abceb0d48564` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:49.504869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0627b19238764d9b8774abceb0d48564)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:49.512101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:49.671506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0627b19238764d9b8774abceb0d48564', 'request_id': 'req-dc809749-0bb1-4561-92ab-d55e89c88855'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:49.684542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0627b19238764d9b8774abceb0d48564', 'request_id': 'req-dc809749-0bb1-4561-92ab-d55e89c88855'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:49.693014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0627b19238764d9b8774abceb0d48564', 'request_id': 'req-dc809749-0bb1-4561-92ab-d55e89c88855'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:49.789382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] The token cache is being invalidate because project 0627b19238764d9b8774abceb0d48564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:49.789686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:49.800680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:49.800680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 731/1449] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:49 2022] DELETE /identity/v3/projects/0627b19238764d9b8774abceb0d48564 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:50.780704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:50.784880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:50.886581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0e02c-8213-44e6-b589-d0c7ffbc7666 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:50.913748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:50.978896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:50.983488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:50.983488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:50.983488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] PATH_INFO: `/v3/users/24da5700a7554b59af438bbc62eb393a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:50.983488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=24da5700a7554b59af438bbc62eb393a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:50.988509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:51.185253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:51.186739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:51.210039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc809749-0bb1-4561-92ab-d55e89c88855 tempest-VolumeQuotaClassesTest-375892553 tempest-VolumeQuotaClassesTest-375892553-project-admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:51.221460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:51.222461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.222747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.223182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] PATH_INFO: `/v3/users/eed05c0d011f43449e73c384513e03df` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.224965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed05c0d011f43449e73c384513e03df)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:51.227696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:51.273683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24da5700a7554b59af438bbc62eb393a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.296614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24da5700a7554b59af438bbc62eb393a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.298197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24da5700a7554b59af438bbc62eb393a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.309688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24da5700a7554b59af438bbc62eb393a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.310785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8b07c53-5ae8-493c-8662-b9b2c2f0b0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24da5700a7554b59af438bbc62eb393a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.347004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:51.410764 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 719/1450] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:50 2022] DELETE /identity/v3/users/24da5700a7554b59af438bbc62eb393a => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:51.415751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.416244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.418460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.441592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:51.449334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:51.469390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed05c0d011f43449e73c384513e03df'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.470498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed05c0d011f43449e73c384513e03df'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.678269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed05c0d011f43449e73c384513e03df'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.685942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed05c0d011f43449e73c384513e03df'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.704189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c34335c1-bb1f-47f1-8284-d66ea49e0f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed05c0d011f43449e73c384513e03df'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:51.716675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 720/1451] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:51.726541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:51.728963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:51.731860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 732/1452] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:51 2022] DELETE /identity/v3/users/eed05c0d011f43449e73c384513e03df => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:51.736644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb1b64-8a5a-4970-8d8e-8c02f86e15c5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.737592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb1b64-8a5a-4970-8d8e-8c02f86e15c5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.738016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb1b64-8a5a-4970-8d8e-8c02f86e15c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.764622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b5bb1b64-8a5a-4970-8d8e-8c02f86e15c5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:51.766149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b5bb1b64-8a5a-4970-8d8e-8c02f86e15c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:51.796759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee5620-9dfd-4152-a9fb-8f0962111e66 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:51.860948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 733/1453] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:51.881197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d9d34-2266-476c-94ee-4018635687f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:51.886833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d9d34-2266-476c-94ee-4018635687f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.886833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d9d34-2266-476c-94ee-4018635687f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.886833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d9d34-2266-476c-94ee-4018635687f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.896404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb1b64-8a5a-4970-8d8e-8c02f86e15c5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:51.902271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d9d34-2266-476c-94ee-4018635687f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:51.903968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d9d34-2266-476c-94ee-4018635687f0 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:51.914116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb1b64-8a5a-4970-8d8e-8c02f86e15c5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:51.923389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:51.925727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.925727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.925727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.926070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 721/1454] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:51.940766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:51.942784 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:51.988277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 734/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:52.872945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:52.972198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aea69-8ada-47e1-b375-e072c70c6453 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:52.973072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aea69-8ada-47e1-b375-e072c70c6453 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:52.973559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aea69-8ada-47e1-b375-e072c70c6453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:53.002472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-519aea69-8ada-47e1-b375-e072c70c6453 None None] MFA Rules not processed for user `eca696f5dfaa466fa06782c82ea42758`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:53.004674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-519aea69-8ada-47e1-b375-e072c70c6453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:53.100436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:53.101942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:53.110407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 722/1456] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:53.140275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc64b8-f9a4-471b-8ddb-54264a741017 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:53.155382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:53.156169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:53.156588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:53.156921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] PATH_INFO: `/v3/projects/f4ce7ac3000b42ac980d2c1c04a7bd95` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:53.160124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ce7ac3000b42ac980d2c1c04a7bd95)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:53.163947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:53.165076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:53.404380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ce7ac3000b42ac980d2c1c04a7bd95', 'request_id': 'req-59009abd-2047-4a45-943f-b5edb70bf7f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:53.414987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ce7ac3000b42ac980d2c1c04a7bd95', 'request_id': 'req-59009abd-2047-4a45-943f-b5edb70bf7f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:53.423606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ce7ac3000b42ac980d2c1c04a7bd95', 'request_id': 'req-59009abd-2047-4a45-943f-b5edb70bf7f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:53.552686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] The token cache is being invalidate because project f4ce7ac3000b42ac980d2c1c04a7bd95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:53.552981 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:53.554324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-59009abd-2047-4a45-943f-b5edb70bf7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:53.558685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 735/1457] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:53 2022] DELETE /identity/v3/projects/f4ce7ac3000b42ac980d2c1c04a7bd95 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:53.563620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aea69-8ada-47e1-b375-e072c70c6453 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:53.585172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aea69-8ada-47e1-b375-e072c70c6453 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:53.636784 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3d4e-d1b0-4855-842d-b5cb2c521bb4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:53.637074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3d4e-d1b0-4855-842d-b5cb2c521bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:53.637349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3d4e-d1b0-4855-842d-b5cb2c521bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-133e3d4e-d1b0-4855-842d-b5cb2c521bb4 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:10:53.646674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:53.647516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:10:53.655975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 736/1458] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:53 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 21 msecs (HTTP/1.1 400) 5 headers in 186 bytes (3 switches on core 0) Oct 26 11:10:53.661340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3d4e-d1b0-4855-842d-b5cb2c521bb4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:53.667269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-133e3d4e-d1b0-4855-842d-b5cb2c521bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:53.721390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:53.723638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:53.723638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:53.723638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] PATH_INFO: `/v3/users/1a281c8fdcfe4b0db7a8d1daf24eb5f0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:53.726588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a281c8fdcfe4b0db7a8d1daf24eb5f0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:53.728525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:53.793124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3d4e-d1b0-4855-842d-b5cb2c521bb4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:53.819410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:53.819968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:53.820289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:53.820564 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] PATH_INFO: `/v3/projects/277708e8fc534cc4bfb5af23735367da` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:53.822152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277708e8fc534cc4bfb5af23735367da)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:53.824779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:53.975963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a281c8fdcfe4b0db7a8d1daf24eb5f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:53.996940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277708e8fc534cc4bfb5af23735367da', 'request_id': 'req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.004552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277708e8fc534cc4bfb5af23735367da', 'request_id': 'req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.012416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a281c8fdcfe4b0db7a8d1daf24eb5f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.013487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a281c8fdcfe4b0db7a8d1daf24eb5f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.014449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277708e8fc534cc4bfb5af23735367da', 'request_id': 'req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.030290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a281c8fdcfe4b0db7a8d1daf24eb5f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.037304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f45ffa79-a2e8-4e1c-9950-113f4fabeff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a281c8fdcfe4b0db7a8d1daf24eb5f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.120233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] The token cache is being invalidate because project 277708e8fc534cc4bfb5af23735367da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:54.120233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.120766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.128395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 737/1459] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:53 2022] DELETE /identity/v3/projects/277708e8fc534cc4bfb5af23735367da => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:54.130438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:54.133039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:54.273783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a1cb2-cdd8-4e25-955c-cc3da89429c5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:54.275752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 723/1460] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:53 2022] DELETE /identity/v3/users/1a281c8fdcfe4b0db7a8d1daf24eb5f0 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:54.368460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:54.394326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:54.395041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:54.395322 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:54.395671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] PATH_INFO: `/v3/users/fd42749682e3439a89b58113bf6e2b54` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:54.397628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd42749682e3439a89b58113bf6e2b54)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:54.399637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:54.563949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:54.660633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd42749682e3439a89b58113bf6e2b54'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.661827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd42749682e3439a89b58113bf6e2b54'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.777291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd42749682e3439a89b58113bf6e2b54'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.778224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd42749682e3439a89b58113bf6e2b54'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.787980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c93c6573-19a4-424a-a302-e1c59e25d538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd42749682e3439a89b58113bf6e2b54'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:54.812213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 738/1461] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:54 2022] DELETE /identity/v3/users/fd42749682e3439a89b58113bf6e2b54 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:55.246080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:55.554056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a0568-38f7-417a-93f8-0d19d99e1d3b None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:55.554332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a0568-38f7-417a-93f8-0d19d99e1d3b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:55.554886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a0568-38f7-417a-93f8-0d19d99e1d3b None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:55.559685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 724/1462] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 26 11:10:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:55.568978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:55.569909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:55.572654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:55.615897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:55.617430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:55.775306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:55.779999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:55.780495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:55.780955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:55.791021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 739/1463] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 26 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:55.819931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] MFA Rules not processed for user `f4c15ef6162348f9b1801c15b959b733`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:55.821494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:55.845296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:55.846997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:55.878740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdf6e7-10eb-4911-b0b3-457938c4c654 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:55.888829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:55.913049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c154983aff14d9895fb4b542db61e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:55.917088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:55.917088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:55.917762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:55.971878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 725/1464] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:55.975473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:55.977124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:55.990048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 740/1465] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:56.112116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:56.113632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:56.239095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0a8d6-7880-488a-9a93-898d509bf85d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:56.254050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:56.255510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:56.256130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:56.257208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] PATH_INFO: `/v3/users/83e3e4b159c94fdf9a03b72ebf62ac46` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:56.261506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e3e4b159c94fdf9a03b72ebf62ac46)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:56.261506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:56.302693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:56.304467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:56.337695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5dacd-4ae0-4057-af82-cf20d48d771d service placement] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:56.364700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:56.365986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:56.366267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:56.367239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] PATH_INFO: `/v3/projects/b06c6d2db8c0414598760d021f139edf` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:56.368815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06c6d2db8c0414598760d021f139edf)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:56.372330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:56.517994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e3e4b159c94fdf9a03b72ebf62ac46'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.538727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06c6d2db8c0414598760d021f139edf', 'request_id': 'req-f44f1d82-919c-4172-bcdb-992c54335b31'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.547038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06c6d2db8c0414598760d021f139edf', 'request_id': 'req-f44f1d82-919c-4172-bcdb-992c54335b31'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.551059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e3e4b159c94fdf9a03b72ebf62ac46'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.551868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e3e4b159c94fdf9a03b72ebf62ac46'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.558180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06c6d2db8c0414598760d021f139edf', 'request_id': 'req-f44f1d82-919c-4172-bcdb-992c54335b31'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.565232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e3e4b159c94fdf9a03b72ebf62ac46'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.565994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6ff8b370-b182-4759-9f7f-b7f8adb466ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e3e4b159c94fdf9a03b72ebf62ac46'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.634964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] The token cache is being invalidate because project b06c6d2db8c0414598760d021f139edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:56.635624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.636205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f44f1d82-919c-4172-bcdb-992c54335b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:56.638343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 741/1466] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:56 2022] DELETE /identity/v3/projects/b06c6d2db8c0414598760d021f139edf => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:56.643650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ceb91-1b43-439e-90b7-471981c23e09 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:56.643969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ceb91-1b43-439e-90b7-471981c23e09 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:56.644283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ceb91-1b43-439e-90b7-471981c23e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:56.673611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ff1ceb91-1b43-439e-90b7-471981c23e09 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:56.674921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ff1ceb91-1b43-439e-90b7-471981c23e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:56.748300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 726/1467] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:56 2022] DELETE /identity/v3/users/83e3e4b159c94fdf9a03b72ebf62ac46 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:56.790217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 742/1468] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:56.790760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:56.791181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:56.791519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:56.850807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:56.852247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:56.964844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 727/1469] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:56.969733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ceb91-1b43-439e-90b7-471981c23e09 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:56.992777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ceb91-1b43-439e-90b7-471981c23e09 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:57.001925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b1c4a-c5ad-4779-912c-3a6665e297fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:57.003138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1c4a-c5ad-4779-912c-3a6665e297fd admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:57.003669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1c4a-c5ad-4779-912c-3a6665e297fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:57.004215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1c4a-c5ad-4779-912c-3a6665e297fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:57.005972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b1c4a-c5ad-4779-912c-3a6665e297fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:57.006739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b1c4a-c5ad-4779-912c-3a6665e297fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:57.016166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b1c4a-c5ad-4779-912c-3a6665e297fd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:57.023543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:57.032102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:57.059056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c84792-d2f0-4a54-8856-a4fe38404811 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:57.064540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:57.066217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:57.066686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:57.066958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 743/1470] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:57.067914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] PATH_INFO: `/v3/projects/2d166711262f46df919bd33fa02d0671` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:57.068506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56e749-f4a4-46c1-945f-7dbde1c59436 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:57.068795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56e749-f4a4-46c1-945f-7dbde1c59436 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:57.069101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56e749-f4a4-46c1-945f-7dbde1c59436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:57.070116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d166711262f46df919bd33fa02d0671)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:57.072814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8b56e749-f4a4-46c1-945f-7dbde1c59436 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:10:57.089049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:10:57.089867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:10:57.098289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 744/1471] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:10:57 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 30 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:10:57.099754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56e749-f4a4-46c1-945f-7dbde1c59436 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:57.153371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56e749-f4a4-46c1-945f-7dbde1c59436 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:57.166856 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:57.167841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:57.172096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:57.172608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:57.173548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:57.180778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:57.248479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d166711262f46df919bd33fa02d0671', 'request_id': 'req-f4d04757-6541-45d5-9210-2fe1edd0de94'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.260526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d166711262f46df919bd33fa02d0671', 'request_id': 'req-f4d04757-6541-45d5-9210-2fe1edd0de94'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.277882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d166711262f46df919bd33fa02d0671', 'request_id': 'req-f4d04757-6541-45d5-9210-2fe1edd0de94'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.346686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 745/1472] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:57 2022] POST /identity/v3/projects => generated 391 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:57.347869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:57.351575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:57.407761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] The token cache is being invalidate because project 2d166711262f46df919bd33fa02d0671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:57.408030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.408930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.414898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 728/1473] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:57 2022] DELETE /identity/v3/projects/2d166711262f46df919bd33fa02d0671 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:10:57.414898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:57.418201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:57.605251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d2b91-f793-4fbe-8543-f45db1b23925 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:57.615160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:57.616341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:57.616644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:57.616924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] PATH_INFO: `/v3/projects/14a7a0c0ef824bbd928dd8a18e41b65e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:57.621631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a7a0c0ef824bbd928dd8a18e41b65e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:57.622776 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d04757-6541-45d5-9210-2fe1edd0de94 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:57.626145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:57.663441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52b881-eb12-476f-989e-0ed372c6b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:57.665179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b881-eb12-476f-989e-0ed372c6b007 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:57.669354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b881-eb12-476f-989e-0ed372c6b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:57.670777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b881-eb12-476f-989e-0ed372c6b007 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:57.672437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52b881-eb12-476f-989e-0ed372c6b007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:57.676220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52b881-eb12-476f-989e-0ed372c6b007 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:57.811653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a7a0c0ef824bbd928dd8a18e41b65e', 'request_id': 'req-db74eb5c-8d33-4049-8c47-c8261fe64b48'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.820477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a7a0c0ef824bbd928dd8a18e41b65e', 'request_id': 'req-db74eb5c-8d33-4049-8c47-c8261fe64b48'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.839325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a7a0c0ef824bbd928dd8a18e41b65e', 'request_id': 'req-db74eb5c-8d33-4049-8c47-c8261fe64b48'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.858750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 729/1474] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:57 2022] POST /identity/v3/users => generated 296 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:57.860363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f0d6-e4cd-4768-a77d-36d09133ad57 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:57.861060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f0d6-e4cd-4768-a77d-36d09133ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:57.861508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f0d6-e4cd-4768-a77d-36d09133ad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:57.912990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-acd2f0d6-e4cd-4768-a77d-36d09133ad57 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:57.914986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-acd2f0d6-e4cd-4768-a77d-36d09133ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:57.955172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] The token cache is being invalidate because project 14a7a0c0ef824bbd928dd8a18e41b65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:10:57.955605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.957122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:57.960485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 746/1475] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:10:57 2022] DELETE /identity/v3/projects/14a7a0c0ef824bbd928dd8a18e41b65e => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:57.973805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:57.976248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.110437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74eb5c-8d33-4049-8c47-c8261fe64b48 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.116129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:58.117659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.118759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.119447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.120533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:58.122400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:58.140865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 747/1476] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:58.141840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.144423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.146999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 730/1477] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.151666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504608-33a8-4f42-9f08-7de3d34f97d1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.154406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504608-33a8-4f42-9f08-7de3d34f97d1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.154689 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504608-33a8-4f42-9f08-7de3d34f97d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.171082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e90dc9-7db8-4727-84b9-5d10666a0847 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.183502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:58.184451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.184775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.185733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] PATH_INFO: `/v3/users/430a8beb34924e54ad6ba7b8370a110a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.187059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=430a8beb34924e54ad6ba7b8370a110a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:58.187686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e1504608-33a8-4f42-9f08-7de3d34f97d1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:58.189477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e1504608-33a8-4f42-9f08-7de3d34f97d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.189604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:58.198239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 731/1478] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.200761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41040-36c4-4cd3-a236-6e30bde88cee None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.200761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41040-36c4-4cd3-a236-6e30bde88cee None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.200930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41040-36c4-4cd3-a236-6e30bde88cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.232318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-45a41040-36c4-4cd3-a236-6e30bde88cee None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:58.234599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-45a41040-36c4-4cd3-a236-6e30bde88cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.251484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 732/1479] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.252872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a41040-36c4-4cd3-a236-6e30bde88cee None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.263418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a41040-36c4-4cd3-a236-6e30bde88cee None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.269367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c3aea-576d-4808-9dab-8896bded1161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:58.270161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c3aea-576d-4808-9dab-8896bded1161 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.270821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c3aea-576d-4808-9dab-8896bded1161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.271115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c3aea-576d-4808-9dab-8896bded1161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.275810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c3aea-576d-4808-9dab-8896bded1161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:58.275810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c3aea-576d-4808-9dab-8896bded1161 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:58.285781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 733/1480] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:58.287049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5c0b3-169d-46f6-8795-9d95fbd64eac None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.287333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5c0b3-169d-46f6-8795-9d95fbd64eac None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.287818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5c0b3-169d-46f6-8795-9d95fbd64eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.312013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a2a5c0b3-169d-46f6-8795-9d95fbd64eac None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:58.313360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a2a5c0b3-169d-46f6-8795-9d95fbd64eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.331170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 734/1481] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.332276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5c0b3-169d-46f6-8795-9d95fbd64eac None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.345063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5c0b3-169d-46f6-8795-9d95fbd64eac None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.354452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:58.355192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.355473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.368281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.371200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:58.371513 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:58.373240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:58.389732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 735/1482] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:58.389732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.396354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:58.403608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe2594-5ed1-4b36-b367-2c605d8ed760 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.409347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:58.410154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.410368 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.410629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] PATH_INFO: `/v3/projects/94f16b553c95469ca3d7eca676ecf98b/users/203da7abee4441deaf6f77035f1f415b/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.413335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430a8beb34924e54ad6ba7b8370a110a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:58.415290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430a8beb34924e54ad6ba7b8370a110a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:58.429591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f16b553c95469ca3d7eca676ecf98b, user_id=203da7abee4441deaf6f77035f1f415b, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:58.433139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:58.542665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430a8beb34924e54ad6ba7b8370a110a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:58.544863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430a8beb34924e54ad6ba7b8370a110a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:58.552349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 736/1483] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:10:58 2022] PUT /identity/v3/projects/94f16b553c95469ca3d7eca676ecf98b/users/203da7abee4441deaf6f77035f1f415b/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:58.554271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.559082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430a8beb34924e54ad6ba7b8370a110a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:58.562982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.575820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:58.595454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 748/1484] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:58 2022] DELETE /identity/v3/users/430a8beb34924e54ad6ba7b8370a110a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:58.596355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.598077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.689204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacdbab-f351-48b7-a731-78c006f8b6a6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.696477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e6c1-93ee-440b-a639-cde8874642b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:58.698438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e6c1-93ee-440b-a639-cde8874642b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.698912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e6c1-93ee-440b-a639-cde8874642b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.699410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e6c1-93ee-440b-a639-cde8874642b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.700267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e6c1-93ee-440b-a639-cde8874642b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:58.702478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e6c1-93ee-440b-a639-cde8874642b4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:58.711040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e5115-a6c8-4b8c-bcaf-b3ea550b0c82 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.726104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5313b5-e256-4ceb-bc85-ca1c06727c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:58.726929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5313b5-e256-4ceb-bc85-ca1c06727c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.727308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5313b5-e256-4ceb-bc85-ca1c06727c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.727653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5313b5-e256-4ceb-bc85-ca1c06727c11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.729504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5313b5-e256-4ceb-bc85-ca1c06727c11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:10:58.729818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5313b5-e256-4ceb-bc85-ca1c06727c11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:58.731945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5313b5-e256-4ceb-bc85-ca1c06727c11 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:58.757116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 737/1485] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:10:58.757116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc2251-777f-4e4f-b86f-099fbc373d4d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.757116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc2251-777f-4e4f-b86f-099fbc373d4d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.759391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc2251-777f-4e4f-b86f-099fbc373d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.805517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-05bc2251-777f-4e4f-b86f-099fbc373d4d None None] MFA Rules not processed for user `203da7abee4441deaf6f77035f1f415b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:58.807410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-05bc2251-777f-4e4f-b86f-099fbc373d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.838343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 749/1486] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:58.841240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bba6-52c7-4ad4-a845-0871558bf995 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.842467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bba6-52c7-4ad4-a845-0871558bf995 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.842947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bba6-52c7-4ad4-a845-0871558bf995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.872126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3ea9bba6-52c7-4ad4-a845-0871558bf995 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:58.873407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3ea9bba6-52c7-4ad4-a845-0871558bf995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.898514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 738/1487] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.899698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc2251-777f-4e4f-b86f-099fbc373d4d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.911502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc2251-777f-4e4f-b86f-099fbc373d4d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:58.965719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 750/1488] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.969004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a073-9c1a-40c5-903a-5daae0d5c802 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.969513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a073-9c1a-40c5-903a-5daae0d5c802 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.969980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a073-9c1a-40c5-903a-5daae0d5c802 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.982478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 751/1489] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 26 11:10:58 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:58.984022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a073-9c1a-40c5-903a-5daae0d5c802 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.011160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a073-9c1a-40c5-903a-5daae0d5c802 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.016382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.017217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.017673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.018131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.019177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.021157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.021510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.022274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.022880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.023377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.024370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.028793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.141305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 752/1490] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:59.141305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.144286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 739/1491] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/projects => generated 403 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:59.144701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.144982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.146692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.158740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757512b4-9665-4e05-b090-eca885cbd4ed admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.222258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2053e7-bf80-48ac-8f66-445791689bdb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.225588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c528ab-9b9d-4d89-a496-5054b701f46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.226507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c528ab-9b9d-4d89-a496-5054b701f46d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.226885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c528ab-9b9d-4d89-a496-5054b701f46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.227141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.227254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c528ab-9b9d-4d89-a496-5054b701f46d admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.227963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c528ab-9b9d-4d89-a496-5054b701f46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.228068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.234389 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.234389 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] PATH_INFO: `/v3/users/bfa99ee8ae73487da15a4adf72c473cb` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.234566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c528ab-9b9d-4d89-a496-5054b701f46d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.234639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa99ee8ae73487da15a4adf72c473cb)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.234639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.355205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 753/1492] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:10:59 2022] POST /identity/v3/projects => generated 393 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:59.356704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.357017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.357295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.380260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] MFA Rules not processed for user `4c154983aff14d9895fb4b542db61e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:59.381505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.458786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 754/1493] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 26 11:10:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:59.460063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.461538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.483332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa99ee8ae73487da15a4adf72c473cb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:59.497027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa99ee8ae73487da15a4adf72c473cb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:59.497748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa99ee8ae73487da15a4adf72c473cb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:59.511744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa99ee8ae73487da15a4adf72c473cb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:59.512560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa99ee8ae73487da15a4adf72c473cb'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:10:59.533972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991408c-ebcd-4f85-9bc7-60d85e2efcca None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.537872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.538730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.538998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.539267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.541029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.592222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 740/1494] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:10:59 2022] DELETE /identity/v3/users/bfa99ee8ae73487da15a4adf72c473cb => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:10:59.592969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.594632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.610139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.611615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.623475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 755/1495] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:59.624384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.625858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.641464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a1be-4eb1-4fc9-8ef9-a01cf73b64f7 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.702124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca318a-c51e-4632-a073-fdb1b0e05704 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.717573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.718260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.718460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.718728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.719985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.721689 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.737718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 741/1496] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:10:59.738931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.740932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8988c-e79e-4caf-ac51-003a23eec784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.741437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.741687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8988c-e79e-4caf-ac51-003a23eec784 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.742036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8988c-e79e-4caf-ac51-003a23eec784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.743058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8988c-e79e-4caf-ac51-003a23eec784 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.744419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8988c-e79e-4caf-ac51-003a23eec784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.747123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8988c-e79e-4caf-ac51-003a23eec784 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.757560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51bf64-5ae2-43f4-8f48-58956f0cc25a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.766405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.767419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.767976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.768625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.769336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.771457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.845273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 756/1497] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:59 2022] POST /identity/v3/users => generated 302 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:59.846389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4071-7d86-4d7e-bf45-3b6faf47c48d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.846673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4071-7d86-4d7e-bf45-3b6faf47c48d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.846944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab4071-7d86-4d7e-bf45-3b6faf47c48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.869805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-87ab4071-7d86-4d7e-bf45-3b6faf47c48d None None] MFA Rules not processed for user `95eef108b14a4a90a8b617088c904a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:59.871365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-87ab4071-7d86-4d7e-bf45-3b6faf47c48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.876762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 742/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:10:59 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:10:59.877562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.879378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.968477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 757/1499] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:59.970137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab4071-7d86-4d7e-bf45-3b6faf47c48d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.974305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d238abe-35ad-4c54-95a7-0c5e4dc95a23 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.984581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab4071-7d86-4d7e-bf45-3b6faf47c48d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.984775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c154983aff14d9895fb4b542db61e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.985575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.985969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.986337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.988487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.989336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.990056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.990343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.990699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.991351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.993236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.005544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 758/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:00.006654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.011255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.013393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.022415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f13cb-a267-4a78-97d5-8453a998329c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.028703 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.029461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.029747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.030155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.030811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.031326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 743/1501] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:00.038512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.043167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.049558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.067224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 759/1502] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:00.067224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.073681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c527b8-861f-4fdf-8005-89ff763a8ca5 service placement] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.080402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.084933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.084933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.084933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] PATH_INFO: `/v3/projects/a490c158ed0542fa863440b35aa0d44d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.084933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a490c158ed0542fa863440b35aa0d44d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.087738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.090800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b324c2-d6ff-4613-8c5a-177664bd27ba admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.095771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.096514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.097112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.097112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.097737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.099511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.111639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 760/1503] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:00.112975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.124089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0e748-2458-4a9c-a76b-f179fca7f626 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.129023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.129723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.130064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.130349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] PATH_INFO: `/v3/projects/30991ce879874c8ea16d262ede10e92d/users/c2acd3fcf92547e88830af3953b3a812/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.146374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30991ce879874c8ea16d262ede10e92d, user_id=c2acd3fcf92547e88830af3953b3a812, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.149167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.298502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 761/1504] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:00 2022] PUT /identity/v3/projects/30991ce879874c8ea16d262ede10e92d/users/c2acd3fcf92547e88830af3953b3a812/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:00.299462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.310907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a490c158ed0542fa863440b35aa0d44d', 'request_id': 'req-63a45a4a-4007-48cb-87cd-bec600b90bf4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.317514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a490c158ed0542fa863440b35aa0d44d', 'request_id': 'req-63a45a4a-4007-48cb-87cd-bec600b90bf4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.326897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a490c158ed0542fa863440b35aa0d44d', 'request_id': 'req-63a45a4a-4007-48cb-87cd-bec600b90bf4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.367251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29794d46-c66e-4775-bb13-6f3176ff85a2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.431675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.431675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.431675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.431675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.432157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.432157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.442685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 762/1505] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:00.443468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.456940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9f484-f3d1-4854-b284-885ef6e3b215 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.462010 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.462678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.465841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.465841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] PATH_INFO: `/v3/projects/eaeaaa4c9ca54ce1b2882444b9e55040/users/e0ac381249fe49e0b548fa5700343665/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.478641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] The token cache is being invalidate because project a490c158ed0542fa863440b35aa0d44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:00.479263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.480130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-63a45a4a-4007-48cb-87cd-bec600b90bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.488075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 744/1506] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:00 2022] DELETE /identity/v3/projects/a490c158ed0542fa863440b35aa0d44d => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:00.489455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.490205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.490492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.496551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaeaaa4c9ca54ce1b2882444b9e55040, user_id=e0ac381249fe49e0b548fa5700343665, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.500789 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.529064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] MFA Rules not processed for user `c2acd3fcf92547e88830af3953b3a812`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:00.530494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.599186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 745/1507] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:00.600215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.601892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.615558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4d10a-1577-47fc-8a9e-dde28f188a1f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.620805 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef577dd3-3cc6-4e48-9fa9-aa2aa125d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.620927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef577dd3-3cc6-4e48-9fa9-aa2aa125d02d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.621170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef577dd3-3cc6-4e48-9fa9-aa2aa125d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.621442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef577dd3-3cc6-4e48-9fa9-aa2aa125d02d admin admin] PATH_INFO: `/v3/projects/23578d0a56574345b39c0862858e6c27/users/7d6215553d33445bb0e45d7620fdc6dd/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.638895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef577dd3-3cc6-4e48-9fa9-aa2aa125d02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23578d0a56574345b39c0862858e6c27, user_id=7d6215553d33445bb0e45d7620fdc6dd, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.641765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef577dd3-3cc6-4e48-9fa9-aa2aa125d02d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.659595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 763/1508] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:00 2022] PUT /identity/v3/projects/eaeaaa4c9ca54ce1b2882444b9e55040/users/e0ac381249fe49e0b548fa5700343665/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:00.660558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.663167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.729366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3757fa8-5dd3-4968-9688-9894aff75cb7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.734113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.734744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.735021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.735294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] PATH_INFO: `/v3/projects/fc8251e1048c4416a6eb0204c73066ce` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.736461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc8251e1048c4416a6eb0204c73066ce)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.738358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.763380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 746/1509] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:00 2022] PUT /identity/v3/projects/23578d0a56574345b39c0862858e6c27/users/7d6215553d33445bb0e45d7620fdc6dd/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:00.765083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.765439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.765811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.794131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:00.795381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.857750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 747/1510] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:00.858396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.860303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.874430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1882180-ea22-4e5f-b8f2-01ed9848931d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.880492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.881103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.881375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.881638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.882294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.884345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.901802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc8251e1048c4416a6eb0204c73066ce', 'request_id': 'req-27ee6206-6519-474c-9a8b-5a5afc18c7f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.907554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc8251e1048c4416a6eb0204c73066ce', 'request_id': 'req-27ee6206-6519-474c-9a8b-5a5afc18c7f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.913827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc8251e1048c4416a6eb0204c73066ce', 'request_id': 'req-27ee6206-6519-474c-9a8b-5a5afc18c7f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.950497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:00.984969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] The token cache is being invalidate because project fc8251e1048c4416a6eb0204c73066ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:00.985374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.986644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-27ee6206-6519-474c-9a8b-5a5afc18c7f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:00.990878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 748/1512] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:00 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:00.991481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 764/1512] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:00 2022] DELETE /identity/v3/projects/fc8251e1048c4416a6eb0204c73066ce => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:00.991694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:00.992978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc68f-8563-420f-846c-23868de18409 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.993379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc68f-8563-420f-846c-23868de18409 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.993514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.993845 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc68f-8563-420f-846c-23868de18409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-774dc68f-8563-420f-846c-23868de18409 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:11:01.001149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:01.002469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:11:01.005137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 765/1513] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:11:00 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 12 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:11:01.005354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.005707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.006022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.017439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:01.063535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] MFA Rules not processed for user `7d6215553d33445bb0e45d7620fdc6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:01.064876 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.083475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a76d6-4340-4d32-a759-c6dcaa043d2d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.088820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bd77d-a661-4f36-a576-2cc96253aa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.089629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bd77d-a661-4f36-a576-2cc96253aa4a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.089976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bd77d-a661-4f36-a576-2cc96253aa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.090374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bd77d-a661-4f36-a576-2cc96253aa4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.091117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bd77d-a661-4f36-a576-2cc96253aa4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.092943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bd77d-a661-4f36-a576-2cc96253aa4a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:01.104934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 749/1514] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:01.109652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.110010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.110289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.136229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:01.140500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.146802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 766/1515] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:01.148944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:01.152363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.158578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 750/1516] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:01.159592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:01.160984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.177075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c84d4-34b0-4bf8-b410-1d9516b6ccb9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.179755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a6c94-27ab-4d32-b4fb-31faf1544006 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.183678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.184412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.184691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.184963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.185747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.187694 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:01.190222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.190948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.191299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.191644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] PATH_INFO: `/v3/users/7995776b6de54f88bf9150dd1a51879c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.193707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7995776b6de54f88bf9150dd1a51879c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.195722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:01.294034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 767/1517] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:01.295540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:01.312852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1976e-3361-45e0-9cb2-9bc00583572f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.318319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.319884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.320272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.320642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] PATH_INFO: `/v3/projects/eaeaaa4c9ca54ce1b2882444b9e55040/users/e0ac381249fe49e0b548fa5700343665/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.323664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaeaaa4c9ca54ce1b2882444b9e55040, user_id=e0ac381249fe49e0b548fa5700343665, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.327005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:01.350645 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 768/1518] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:01 2022] PUT /identity/v3/projects/eaeaaa4c9ca54ce1b2882444b9e55040/users/e0ac381249fe49e0b548fa5700343665/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:01.360737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:01.362555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.434704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd99e9-9b4c-424f-83da-191d609db74d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.439803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.444321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.444321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.444321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.444321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.444321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:01.458642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7995776b6de54f88bf9150dd1a51879c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:01.486357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7995776b6de54f88bf9150dd1a51879c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:01.487823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7995776b6de54f88bf9150dd1a51879c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:01.503447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7995776b6de54f88bf9150dd1a51879c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:01.504239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7995776b6de54f88bf9150dd1a51879c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:01.551053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 769/1519] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/projects => generated 405 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:01.552319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:01.554899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.601242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 751/1520] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:01 2022] DELETE /identity/v3/users/7995776b6de54f88bf9150dd1a51879c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:01.602385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:01.604675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.685674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d49ad-0a45-4ceb-86c1-945683070c58 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.746827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a1555-96de-4c03-a678-c90b758de0cd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.760219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c19713-41b6-44a2-8387-0b3de085e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.761784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19713-41b6-44a2-8387-0b3de085e6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.761784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19713-41b6-44a2-8387-0b3de085e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.761784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19713-41b6-44a2-8387-0b3de085e6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.762800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c19713-41b6-44a2-8387-0b3de085e6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.764528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c19713-41b6-44a2-8387-0b3de085e6a5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:01.782004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042884c-5cb5-4375-b4e5-33d7ce8a920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.782529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042884c-5cb5-4375-b4e5-33d7ce8a920d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.782814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042884c-5cb5-4375-b4e5-33d7ce8a920d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.783090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042884c-5cb5-4375-b4e5-33d7ce8a920d admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.783762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042884c-5cb5-4375-b4e5-33d7ce8a920d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:01.784178 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042884c-5cb5-4375-b4e5-33d7ce8a920d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.785771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042884c-5cb5-4375-b4e5-33d7ce8a920d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:01.792069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 752/1521] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:01.796471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad7c1-a6f7-4c92-aee8-d7885e0500d0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.804535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad7c1-a6f7-4c92-aee8-d7885e0500d0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.804969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad7c1-a6f7-4c92-aee8-d7885e0500d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.805441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 770/1522] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:01.807672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.807997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.809744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.838996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] MFA Rules not processed for user `eca696f5dfaa466fa06782c82ea42758`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:01.840347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.852872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-493ad7c1-a6f7-4c92-aee8-d7885e0500d0 None None] MFA Rules not processed for user `e0ac381249fe49e0b548fa5700343665`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:01.857600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-493ad7c1-a6f7-4c92-aee8-d7885e0500d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.878826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:01.924501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 771/1523] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:01.925498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:01.926968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.972890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d206d-07cc-42e6-a4e3-ee0103f02009 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:01.988464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:01.989969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.990382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.990720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.991375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:01.993723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.005034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 753/1524] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:02.010492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ad7c1-a6f7-4c92-aee8-d7885e0500d0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.028403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ad7c1-a6f7-4c92-aee8-d7885e0500d0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.036131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.036131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.036131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.036131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] PATH_INFO: `/v3/projects/291fa40fd4174c9cba616089489f6fd9/users/eb824a6e45b54f8688bbd9dd2b52e7d4/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.066240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=291fa40fd4174c9cba616089489f6fd9, user_id=eb824a6e45b54f8688bbd9dd2b52e7d4, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.070540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.079146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:02.157630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 772/1525] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/users => generated 309 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:02.161348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.179192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809b54c-e6a3-4120-afee-9fa9e526cb99 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.185438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.186243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.186522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.186988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.187629 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.190382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.259194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 754/1526] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:02 2022] PUT /identity/v3/projects/291fa40fd4174c9cba616089489f6fd9/users/eb824a6e45b54f8688bbd9dd2b52e7d4/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:02.259194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.261407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:02.326317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6320fcd-99a7-4731-be27-a8979263e81c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.331084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.331707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.332069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.332350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.414468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 773/1527] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:02 2022] POST /identity/v3/projects => generated 381 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:02.415581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.422538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.424395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.453472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 755/1528] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:02.454451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.550172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36684d-9ff4-4fe1-bdc5-7c70abc03243 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.564132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.564816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.565105 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.565475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.566446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.578166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.592650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b49923-0fe4-46a3-b818-9f099ceb56e1 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.598483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.599110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.599390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.599669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.600368 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.602361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.617113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 774/1529] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:02.621335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.639156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 756/1530] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:02.646987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.647690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5cf8d-8d67-4787-9a29-6724b78f647f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.648968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:02.659779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.661994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.661994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.661994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.661994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.663469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.732967 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5fc77-4405-47ed-9fa7-3ef915bc08d7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.746286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.747196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.747823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.748093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.753860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:02.795264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 775/1531] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:02 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:02.795264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.812442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2ff65-939e-4e8e-a1b9-ef0bf3e9e146 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.817608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.819103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.819103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.819103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] PATH_INFO: `/v3/projects/3f1f181cfff64dcba1d5c138b4e22c62/users/aae5567c31454c1e9472528024a10708/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.854627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1f181cfff64dcba1d5c138b4e22c62, user_id=aae5567c31454c1e9472528024a10708, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.854627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.886673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.888325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.900039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 757/1532] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:02.914598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.937078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fffae-ad74-4257-a8df-657410e0ab4a service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.942870 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a69f2-f5f6-42e1-ae9c-53d3ca2cd30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.943688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a69f2-f5f6-42e1-ae9c-53d3ca2cd30b admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.944093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a69f2-f5f6-42e1-ae9c-53d3ca2cd30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.944472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a69f2-f5f6-42e1-ae9c-53d3ca2cd30b admin admin] PATH_INFO: `/v3/projects/291fa40fd4174c9cba616089489f6fd9/users/eb824a6e45b54f8688bbd9dd2b52e7d4/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.946731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a69f2-f5f6-42e1-ae9c-53d3ca2cd30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=291fa40fd4174c9cba616089489f6fd9, user_id=eb824a6e45b54f8688bbd9dd2b52e7d4, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.950436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a69f2-f5f6-42e1-ae9c-53d3ca2cd30b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.966620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 776/1533] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:02 2022] PUT /identity/v3/projects/3f1f181cfff64dcba1d5c138b4e22c62/users/aae5567c31454c1e9472528024a10708/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:02.967653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.992602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 758/1534] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:02 2022] PUT /identity/v3/projects/291fa40fd4174c9cba616089489f6fd9/users/eb824a6e45b54f8688bbd9dd2b52e7d4/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:02.995288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad5aa7-5e59-42a3-a01c-33c5404ff9c0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.995288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad5aa7-5e59-42a3-a01c-33c5404ff9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.995549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad5aa7-5e59-42a3-a01c-33c5404ff9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a4ad5aa7-5e59-42a3-a01c-33c5404ff9c0 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:11:03.003568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:03.004530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 11:11:03.026271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 759/1535] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:11:02 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 27 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:11:03.026271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad5aa7-5e59-42a3-a01c-33c5404ff9c0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.058121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:03.138358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc042d6f-e2b2-4901-bece-a9acc7d8c461 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.143636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ad728-e2fc-4ada-8267-07d1c18d8e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.144477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad728-e2fc-4ada-8267-07d1c18d8e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.144790 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad728-e2fc-4ada-8267-07d1c18d8e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.145408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad728-e2fc-4ada-8267-07d1c18d8e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.145894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ad728-e2fc-4ada-8267-07d1c18d8e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:03.146551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad5aa7-5e59-42a3-a01c-33c5404ff9c0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.147754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ad728-e2fc-4ada-8267-07d1c18d8e31 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:03.153406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.154257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.154743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.155367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.156329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:03.158835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:03.162612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 777/1536] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:03.168776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5e3c-8198-4d1a-9314-8371b8ab631e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.170269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5e3c-8198-4d1a-9314-8371b8ab631e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.170622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5e3c-8198-4d1a-9314-8371b8ab631e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.177093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 760/1537] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:03.180706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.195672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f1704-c52e-4d91-9a3f-f149e00c3d75 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.201901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.202688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.203044 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.203384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.204338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:03.204547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b05a5e3c-8198-4d1a-9314-8371b8ab631e None None] MFA Rules not processed for user `eb824a6e45b54f8688bbd9dd2b52e7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:03.205885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b05a5e3c-8198-4d1a-9314-8371b8ab631e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:03.206089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:03.219101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 761/1538] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:03.220087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.232765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf332d-8846-4eba-843a-306072af014a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.238867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.239682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.240065 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.241083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] PATH_INFO: `/v3/projects/3f1f181cfff64dcba1d5c138b4e22c62/users/aae5567c31454c1e9472528024a10708/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.243540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1f181cfff64dcba1d5c138b4e22c62, user_id=aae5567c31454c1e9472528024a10708, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:03.246997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:03.268059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 762/1539] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:03 2022] PUT /identity/v3/projects/3f1f181cfff64dcba1d5c138b4e22c62/users/aae5567c31454c1e9472528024a10708/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:03.269349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.336566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 778/1540] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:03.338919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecef2c7-b6a0-4295-af0e-488968be5156 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.339430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecef2c7-b6a0-4295-af0e-488968be5156 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.339699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecef2c7-b6a0-4295-af0e-488968be5156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.377401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73c064-d76f-4de1-9855-69e0e232edfb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.384098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8ecef2c7-b6a0-4295-af0e-488968be5156 None None] MFA Rules not processed for user `aae5567c31454c1e9472528024a10708`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:03.385955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8ecef2c7-b6a0-4295-af0e-488968be5156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:03.404196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961228-6f7a-40f2-9f24-bac328f7ff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.404886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961228-6f7a-40f2-9f24-bac328f7ff54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.405170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961228-6f7a-40f2-9f24-bac328f7ff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.405445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961228-6f7a-40f2-9f24-bac328f7ff54 admin admin] PATH_INFO: `/v3/projects/d03d408d29084dcb99cf7558556f9df6/users/0af534bcdd5b44cfb010134ae8b682da/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.409013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:03.432545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961228-6f7a-40f2-9f24-bac328f7ff54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03d408d29084dcb99cf7558556f9df6, user_id=0af534bcdd5b44cfb010134ae8b682da, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:03.436436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961228-6f7a-40f2-9f24-bac328f7ff54 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:03.500112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 779/1541] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:03.500112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecef2c7-b6a0-4295-af0e-488968be5156 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.543608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 763/1542] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:03 2022] PUT /identity/v3/projects/d03d408d29084dcb99cf7558556f9df6/users/0af534bcdd5b44cfb010134ae8b682da/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:03.554486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.554768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.555918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.589748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] MFA Rules not processed for user `0af534bcdd5b44cfb010134ae8b682da`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:03.591871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:03.639109 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecef2c7-b6a0-4295-af0e-488968be5156 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.678772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 764/1543] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:03.693022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.694535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:03.709568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.710224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.710507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.710782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.803230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:03.805660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:03.820311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 780/1544] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:03.845717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98aa9b-c026-4cf3-89ee-f53a83c28fbc None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.863425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.870512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.871398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.872568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.873089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.876797 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7dec-f4b4-45a2-a1bf-3a2b8d6986b1 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:03.882133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8079d7a-99f1-4640-bc68-c706709f83f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:03.882901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079d7a-99f1-4640-bc68-c706709f83f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:03.883437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079d7a-99f1-4640-bc68-c706709f83f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:03.883998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079d7a-99f1-4640-bc68-c706709f83f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:03.902413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:03.907149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:03.936708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 765/1545] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:03.937986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:03.940059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:04.089107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eca74-6936-47f6-824f-29be716ee69e service nova] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:04.103942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8079d7a-99f1-4640-bc68-c706709f83f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:04.103942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8079d7a-99f1-4640-bc68-c706709f83f4 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:04.105219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:04.105219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.105219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.105219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] PATH_INFO: `/v3/projects/ed66b79140564481a906f7b8e5f301a2` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.105709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed66b79140564481a906f7b8e5f301a2)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:04.105709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:04.116946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 781/1546] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:03 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:04.157893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:04.284593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed66b79140564481a906f7b8e5f301a2', 'request_id': 'req-059aac91-14b7-47d7-b437-48c964281d64'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:04.293387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed66b79140564481a906f7b8e5f301a2', 'request_id': 'req-059aac91-14b7-47d7-b437-48c964281d64'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:04.302600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed66b79140564481a906f7b8e5f301a2', 'request_id': 'req-059aac91-14b7-47d7-b437-48c964281d64'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:04.409601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:04.415022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] The token cache is being invalidate because project ed66b79140564481a906f7b8e5f301a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:04.427768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:04.427768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-059aac91-14b7-47d7-b437-48c964281d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:04.427768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 766/1547] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:03 2022] DELETE /identity/v3/projects/ed66b79140564481a906f7b8e5f301a2 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:04.428419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4398b3-4fad-4ea6-a668-dead8854235e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.428419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4398b3-4fad-4ea6-a668-dead8854235e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.428419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4398b3-4fad-4ea6-a668-dead8854235e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.465167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ee4398b3-4fad-4ea6-a668-dead8854235e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:04.466580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ee4398b3-4fad-4ea6-a668-dead8854235e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:04.572932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 782/1548] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:04.582393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.582393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.582393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.609652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:04.611201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:04.628632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 767/1549] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:04.640380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4398b3-4fad-4ea6-a668-dead8854235e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:04.671302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:04.672968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4398b3-4fad-4ea6-a668-dead8854235e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:04.678731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:04.692970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:04.693609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.693895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.694244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.694823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:04.695131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:04.696852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:04.724869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f84ac8-e06c-4d97-a857-fae4b6493e1e None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:04.735536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:04.736245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.736662 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.738760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] PATH_INFO: `/v3/users/7d6215553d33445bb0e45d7620fdc6dd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.740536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 783/1550] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:04.745863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:04.747280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6215553d33445bb0e45d7620fdc6dd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:04.752506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:04.783251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cca60-c46c-43eb-860a-01b2fa6cd1f4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:04.799495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:04.802515 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.803183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.803668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.810722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:04.819364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:04.985101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 784/1551] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:04 2022] POST /identity/v3/projects => generated 383 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:04.985101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:05.111429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03e716-5f44-46fd-8732-27ba398ff320 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:05.118606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:05.120290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.120625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.120878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.121565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:05.125011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:05.148914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6215553d33445bb0e45d7620fdc6dd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:05.193110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6215553d33445bb0e45d7620fdc6dd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:05.193835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6215553d33445bb0e45d7620fdc6dd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:05.210506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6215553d33445bb0e45d7620fdc6dd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:05.213130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4fc48534-cc02-4302-960a-4fee84ea430b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6215553d33445bb0e45d7620fdc6dd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:05.287862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 785/1552] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:04 2022] POST /identity/v3/users => generated 292 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:05.297897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:05.299962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:05.325615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 768/1553] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:04 2022] DELETE /identity/v3/users/7d6215553d33445bb0e45d7620fdc6dd => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:05.332165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.332453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.332828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.334278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11855f5-2057-4087-a112-8e4d6d8cd836 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:05.396924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:05.398170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:05.484349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:05.485262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.485648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.486047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.487198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:05.489233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:05.506772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 786/1554] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:05.513406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:05.537006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840c3dc-a8c2-4235-8593-19b3727a1a5b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:05.550033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:05.550275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.553287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.554409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.554409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:05.557623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:05.586264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 787/1555] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:05.592331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:05.593542 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 769/1556] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:05.611914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:05.613846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:05.617843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b79295-5441-433d-b187-a2ad01cd2ad0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:05.630100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:05.631071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.631542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.632054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] PATH_INFO: `/v3/projects/38ba714da82e44d8948acc0b148891a8/users/be9483d392294745807564c4b5824077/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.674187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ba714da82e44d8948acc0b148891a8, user_id=be9483d392294745807564c4b5824077, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:05.678927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:05.725698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc41d2-741e-45c9-8b3e-46ea39fbf077 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:05.734679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438fec4-1f5b-4dab-a732-e80fb5a0e596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:05.735830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438fec4-1f5b-4dab-a732-e80fb5a0e596 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.736259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438fec4-1f5b-4dab-a732-e80fb5a0e596 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.736696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438fec4-1f5b-4dab-a732-e80fb5a0e596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.753065 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438fec4-1f5b-4dab-a732-e80fb5a0e596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:05.754838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438fec4-1f5b-4dab-a732-e80fb5a0e596 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:05.773539 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 770/1557] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:05.781444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 788/1558] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:05 2022] PUT /identity/v3/projects/38ba714da82e44d8948acc0b148891a8/users/be9483d392294745807564c4b5824077/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:05.793083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9ba2-5ec1-4d17-ab67-c8ad80e20b14 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.793697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9ba2-5ec1-4d17-ab67-c8ad80e20b14 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.794211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9ba2-5ec1-4d17-ab67-c8ad80e20b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.895858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7b1f9ba2-5ec1-4d17-ab67-c8ad80e20b14 None None] MFA Rules not processed for user `be9483d392294745807564c4b5824077`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:05.897088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7b1f9ba2-5ec1-4d17-ab67-c8ad80e20b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:06.015830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 771/1559] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:06.044696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:06.051579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:06.160090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928d575-6095-41c0-9b59-da65558bf335 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:06.169399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:06.171878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:06.172182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:06.172463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:06.194844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:06.200043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:06.223656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 789/1560] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:06.252431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f9ba2-5ec1-4d17-ab67-c8ad80e20b14 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:06.270970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:06.272546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:06.387990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f9ba2-5ec1-4d17-ab67-c8ad80e20b14 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:06.395009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:06.395681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:06.396615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:06.396903 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] PATH_INFO: `/v3/users/be9483d392294745807564c4b5824077` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:06.398446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be9483d392294745807564c4b5824077)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:06.400736 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:06.410064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d87cd4-df10-43fd-98dc-e227a59c4eae service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:06.419358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:06.420044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:06.420321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:06.420594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] PATH_INFO: `/v3/users/c2acd3fcf92547e88830af3953b3a812` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:06.422433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2acd3fcf92547e88830af3953b3a812)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:06.425489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:06.437849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:06.679074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be9483d392294745807564c4b5824077'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.680491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2acd3fcf92547e88830af3953b3a812'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.686517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2acd3fcf92547e88830af3953b3a812'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.723875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:06.752379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be9483d392294745807564c4b5824077'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.753617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be9483d392294745807564c4b5824077'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.771222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be9483d392294745807564c4b5824077'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.772270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a4f88d13-1cf5-4c39-b77a-e0697e7eea41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be9483d392294745807564c4b5824077'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.809067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2acd3fcf92547e88830af3953b3a812'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.810306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2acd3fcf92547e88830af3953b3a812'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.828635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2acd3fcf92547e88830af3953b3a812'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:06.858554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 790/1561] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:06 2022] DELETE /identity/v3/users/c2acd3fcf92547e88830af3953b3a812 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:06.859766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:06.861759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:06.951926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 772/1562] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:06 2022] DELETE /identity/v3/users/be9483d392294745807564c4b5824077 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:06.956033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b843-5ca0-402d-86c8-6cfaf6488f40 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:06.957052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b843-5ca0-402d-86c8-6cfaf6488f40 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:06.957624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b843-5ca0-402d-86c8-6cfaf6488f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:07.002524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5fe3b843-5ca0-402d-86c8-6cfaf6488f40 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:07.004188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5fe3b843-5ca0-402d-86c8-6cfaf6488f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:07.021204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ae0b7-f35e-42d3-8291-a84b1d7fb537 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:07.091954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 773/1563] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:07.091954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:07.091954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:07.091954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:07.101747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:07.102916 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:07.104800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:07.105267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] PATH_INFO: `/v3/projects/23578d0a56574345b39c0862858e6c27` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:07.107001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23578d0a56574345b39c0862858e6c27)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:07.109275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:07.112703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:07.114453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:07.136978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 774/1564] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:07.137922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:07.139382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:07.247161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818b529-c61a-4fba-b7a7-2c665bf33279 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:07.251167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:07.252518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:07.252976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:07.253394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:07.267669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:07.269383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:07.284848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 775/1565] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:07.285244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:07.297451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906df45f-f8a7-43da-b540-13f6ca0544fd service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:07.302519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:07.303358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:07.304180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:07.315824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:07.332071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:07.333425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23578d0a56574345b39c0862858e6c27', 'request_id': 'req-773c0af2-6fba-4701-830a-42c9c976a1b0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.335841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:07.346773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23578d0a56574345b39c0862858e6c27', 'request_id': 'req-773c0af2-6fba-4701-830a-42c9c976a1b0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.363579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23578d0a56574345b39c0862858e6c27', 'request_id': 'req-773c0af2-6fba-4701-830a-42c9c976a1b0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.366642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 776/1566] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:07.391421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:07.393666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:07.419969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843847b-788d-4c32-9238-41623c9ae478 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:07.502817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] The token cache is being invalidate because project 23578d0a56574345b39c0862858e6c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:07.503579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.504644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.512586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 791/1567] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:06 2022] DELETE /identity/v3/projects/23578d0a56574345b39c0862858e6c27 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:07.514410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:07.518422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:07.518956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:07.519754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:07.520509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:07.520797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] PATH_INFO: `/v3/users/f4c15ef6162348f9b1801c15b959b733` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:07.522723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c15ef6162348f9b1801c15b959b733)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:07.525975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:07.542228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c0af2-6fba-4701-830a-42c9c976a1b0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:07.551638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:07.553728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:07.554337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:07.554835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] PATH_INFO: `/v3/users/aae5567c31454c1e9472528024a10708` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:07.560938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae5567c31454c1e9472528024a10708)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:07.565936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:07.796860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c15ef6162348f9b1801c15b959b733'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.813094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c15ef6162348f9b1801c15b959b733'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.813999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c15ef6162348f9b1801c15b959b733'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.821629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c15ef6162348f9b1801c15b959b733'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.823428 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c15ef6162348f9b1801c15b959b733'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.873290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae5567c31454c1e9472528024a10708'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.874101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae5567c31454c1e9472528024a10708'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:07.929702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 777/1568] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:07 2022] DELETE /identity/v3/users/f4c15ef6162348f9b1801c15b959b733 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:08.020071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae5567c31454c1e9472528024a10708'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.020930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae5567c31454c1e9472528024a10708'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.029442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae5567c31454c1e9472528024a10708'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.059102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 792/1569] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:07 2022] DELETE /identity/v3/users/aae5567c31454c1e9472528024a10708 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:08.230982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:08.232702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:08.359932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:08.361408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:08.428660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcd60e-4fe4-4ed9-95c9-73c620a0177b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:08.517438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c694ec-4d3b-4751-b8ea-467e3e2d1b68 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:08.519628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:08.520322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:08.520652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:08.520933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] PATH_INFO: `/v3/projects/30991ce879874c8ea16d262ede10e92d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:08.522307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30991ce879874c8ea16d262ede10e92d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:08.524408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:08.529203 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:08.531570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:08.532120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:08.532387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] PATH_INFO: `/v3/projects/38ba714da82e44d8948acc0b148891a8` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:08.538579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ba714da82e44d8948acc0b148891a8)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:08.540198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:08.730672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:08.740532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30991ce879874c8ea16d262ede10e92d', 'request_id': 'req-9d37ba90-5a76-4e1b-9547-083d7c242e38'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.748372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30991ce879874c8ea16d262ede10e92d', 'request_id': 'req-9d37ba90-5a76-4e1b-9547-083d7c242e38'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.760772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30991ce879874c8ea16d262ede10e92d', 'request_id': 'req-9d37ba90-5a76-4e1b-9547-083d7c242e38'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.872362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] The token cache is being invalidate because project 30991ce879874c8ea16d262ede10e92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:08.873441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.874464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.876606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 778/1570] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:08 2022] DELETE /identity/v3/projects/30991ce879874c8ea16d262ede10e92d => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:08.880309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:08.883057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ba714da82e44d8948acc0b148891a8', 'request_id': 'req-0d6000a9-c302-4263-8446-0dc62579a731'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.885643 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:08.898661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ba714da82e44d8948acc0b148891a8', 'request_id': 'req-0d6000a9-c302-4263-8446-0dc62579a731'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.916636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ba714da82e44d8948acc0b148891a8', 'request_id': 'req-0d6000a9-c302-4263-8446-0dc62579a731'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:08.992512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37ba90-5a76-4e1b-9547-083d7c242e38 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:09.031269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] The token cache is being invalidate because project 38ba714da82e44d8948acc0b148891a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:09.031814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.039445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.041949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 793/1571] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:08 2022] DELETE /identity/v3/projects/38ba714da82e44d8948acc0b148891a8 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:09.042999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:09.046745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:09.067866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:09.071375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:09.077847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:09.077847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] PATH_INFO: `/v3/projects/b68088f7836d42eab080dd159a453ca9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:09.084753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68088f7836d42eab080dd159a453ca9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:09.086856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:09.182607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6000a9-c302-4263-8446-0dc62579a731 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:09.188765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:09.189544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:09.189883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:09.190222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] PATH_INFO: `/v3/users/eb824a6e45b54f8688bbd9dd2b52e7d4` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:09.191585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb824a6e45b54f8688bbd9dd2b52e7d4)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:09.194006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:09.351927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68088f7836d42eab080dd159a453ca9', 'request_id': 'req-620903f8-69da-40f6-b31e-f7c217b17056'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.357895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68088f7836d42eab080dd159a453ca9', 'request_id': 'req-620903f8-69da-40f6-b31e-f7c217b17056'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.364188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68088f7836d42eab080dd159a453ca9', 'request_id': 'req-620903f8-69da-40f6-b31e-f7c217b17056'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.577058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:09.583742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] The token cache is being invalidate because project b68088f7836d42eab080dd159a453ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:09.584590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.585486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-620903f8-69da-40f6-b31e-f7c217b17056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.587221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 779/1572] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:08 2022] DELETE /identity/v3/projects/b68088f7836d42eab080dd159a453ca9 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:09.589782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:09.631639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:09.631639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:09.632328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb824a6e45b54f8688bbd9dd2b52e7d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.632328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb824a6e45b54f8688bbd9dd2b52e7d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.632847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:09.632847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:09.676323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 780/1573] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:09.677176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:09.678442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:09.690331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f30e0-d5f3-4231-93ec-e1e861ebcc11 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:09.715794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:09.715794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:09.715794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:09.715794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] PATH_INFO: `/v3/projects/3f1f181cfff64dcba1d5c138b4e22c62` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:09.716175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1f181cfff64dcba1d5c138b4e22c62)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:09.716175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:09.884533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb824a6e45b54f8688bbd9dd2b52e7d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.885764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb824a6e45b54f8688bbd9dd2b52e7d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.899315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb824a6e45b54f8688bbd9dd2b52e7d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.927536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1f181cfff64dcba1d5c138b4e22c62', 'request_id': 'req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.927536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1f181cfff64dcba1d5c138b4e22c62', 'request_id': 'req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.928112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 794/1574] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:09 2022] DELETE /identity/v3/users/eb824a6e45b54f8688bbd9dd2b52e7d4 => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:09.928112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:09.928256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1f181cfff64dcba1d5c138b4e22c62', 'request_id': 'req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:09.928333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:09.991626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c00e5-2860-4fb8-8c38-e6f5aac7e63f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:09.996620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:10.009465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.009465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.009465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] PATH_INFO: `/v3/users/95eef108b14a4a90a8b617088c904a08` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.009465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=95eef108b14a4a90a8b617088c904a08)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:10.009465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:10.050276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] The token cache is being invalidate because project 3f1f181cfff64dcba1d5c138b4e22c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:10.231297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:10.231297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-2d8a5c78-b23a-4a8b-a662-9c4cbdaee146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:10.231297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 781/1575] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:09 2022] DELETE /identity/v3/projects/3f1f181cfff64dcba1d5c138b4e22c62 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:10.231297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cea55-5e91-4738-aafd-2cd8bb300849 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.231297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cea55-5e91-4738-aafd-2cd8bb300849 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.231297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cea55-5e91-4738-aafd-2cd8bb300849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.232358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-781cea55-5e91-4738-aafd-2cd8bb300849 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:10.232358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-781cea55-5e91-4738-aafd-2cd8bb300849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:10.232358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 782/1576] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:10.232358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0833-c6c4-4abe-bab7-e3086bfd4e3d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.232358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0833-c6c4-4abe-bab7-e3086bfd4e3d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.232358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c0833-c6c4-4abe-bab7-e3086bfd4e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.232820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-384c0833-c6c4-4abe-bab7-e3086bfd4e3d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:10.232820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-384c0833-c6c4-4abe-bab7-e3086bfd4e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:10.232820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 783/1577] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:10.232820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be3562-b9e1-4d07-a9ae-364bb0389696 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.232820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be3562-b9e1-4d07-a9ae-364bb0389696 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.232820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be3562-b9e1-4d07-a9ae-364bb0389696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.233254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-39be3562-b9e1-4d07-a9ae-364bb0389696 None None] MFA Rules not processed for user `eca696f5dfaa466fa06782c82ea42758`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:10.233254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-39be3562-b9e1-4d07-a9ae-364bb0389696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:10.233254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 784/1578] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:10.233254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b39fc1-19a8-4edf-ab15-dc7e841da508 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.233254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b39fc1-19a8-4edf-ab15-dc7e841da508 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.233254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b39fc1-19a8-4edf-ab15-dc7e841da508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.238690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-14b39fc1-19a8-4edf-ab15-dc7e841da508 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:10.239927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-14b39fc1-19a8-4edf-ab15-dc7e841da508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:10.248382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 785/1579] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:10.256249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfaf64-3e7e-49eb-beaa-16b640fda2e0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.256531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfaf64-3e7e-49eb-beaa-16b640fda2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.256803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfaf64-3e7e-49eb-beaa-16b640fda2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.298049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-22bfaf64-3e7e-49eb-beaa-16b640fda2e0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:10.298049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-22bfaf64-3e7e-49eb-beaa-16b640fda2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:10.298049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 786/1580] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:10.298049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfaf64-3e7e-49eb-beaa-16b640fda2e0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:10.299990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfaf64-3e7e-49eb-beaa-16b640fda2e0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:10.304669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176eee7-a887-4410-81de-f8eb7225a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:10.305273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176eee7-a887-4410-81de-f8eb7225a3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.305553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176eee7-a887-4410-81de-f8eb7225a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.305826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176eee7-a887-4410-81de-f8eb7225a3b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.306474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176eee7-a887-4410-81de-f8eb7225a3b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:10.307342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176eee7-a887-4410-81de-f8eb7225a3b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:10.309079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176eee7-a887-4410-81de-f8eb7225a3b0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:10.320657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 787/1581] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:10.321943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a241ae-c978-47c5-88ab-1ba0d2773376 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.322240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a241ae-c978-47c5-88ab-1ba0d2773376 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.322611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a241ae-c978-47c5-88ab-1ba0d2773376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.342261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-72a241ae-c978-47c5-88ab-1ba0d2773376 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:10.343467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-72a241ae-c978-47c5-88ab-1ba0d2773376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:10.351598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 788/1582] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:10.352394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a241ae-c978-47c5-88ab-1ba0d2773376 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:10.362391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a241ae-c978-47c5-88ab-1ba0d2773376 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:10.367009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:10.367609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.367915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.368190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.368834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:10.369197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:10.370815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:10.382486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 789/1583] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:10.383342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:10.392887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6723-2b31-4f93-a57c-eb1b8e08ed5c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:10.397682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:10.398267 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:10.398549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:10.398826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:10.399458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:10.401917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.041926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 790/1584] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:10 2022] POST /identity/v3/projects => generated 375 bytes in 1657 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:12.041926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:12.090840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95eef108b14a4a90a8b617088c904a08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.092142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95eef108b14a4a90a8b617088c904a08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.117351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3f2a-c301-4796-b031-422927e1e028 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:12.123428 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:12.124077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.124356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.124627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.125290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:12.125595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.127535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.142127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 791/1585] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:12.142924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:12.156409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6284a42-fba5-4737-aa79-6a2dc6431f9c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:12.163557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:12.164224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.164494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.164772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.165402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.167877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.246374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95eef108b14a4a90a8b617088c904a08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.246963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95eef108b14a4a90a8b617088c904a08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.256498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-619c867b-24e3-45b5-a093-ea0ee9054edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95eef108b14a4a90a8b617088c904a08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.282973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 795/1586] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:09 2022] DELETE /identity/v3/users/95eef108b14a4a90a8b617088c904a08 => generated 0 bytes in 2364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:12.284355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b280c7-ab67-4e2f-ab39-c1867157f586 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.284638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b280c7-ab67-4e2f-ab39-c1867157f586 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.285012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b280c7-ab67-4e2f-ab39-c1867157f586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-28b280c7-ab67-4e2f-ab39-c1867157f586 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:11:12.295559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:12.304848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:11:12.308225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 796/1587] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:11:12 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 13 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:11:12.308225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b280c7-ab67-4e2f-ab39-c1867157f586 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:12.308225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-28b280c7-ab67-4e2f-ab39-c1867157f586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:12.320749 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 792/1588] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:12 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:12.321290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:12.325728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:12.422768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84281c3-a093-4ecf-9222-a585a7e33cb3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:12.424839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b280c7-ab67-4e2f-ab39-c1867157f586 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:12.430577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:12.432153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.432558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.432983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.433631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.433840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:12.434513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.434794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.435698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] PATH_INFO: `/v3/users/203da7abee4441deaf6f77035f1f415b` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.436906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=203da7abee4441deaf6f77035f1f415b)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.438980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.439923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.572843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 793/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:12 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:12.572843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:12.574903 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:12.588399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfae9e-c9d9-4913-9b0c-2c6328fe8df8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:12.593578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:12.594177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.594466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.594861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.595630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.597873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.699882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '203da7abee4441deaf6f77035f1f415b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.700676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '203da7abee4441deaf6f77035f1f415b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.720433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 794/1590] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:12 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:12.720433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:12.726274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:12.779822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '203da7abee4441deaf6f77035f1f415b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.780773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '203da7abee4441deaf6f77035f1f415b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.789034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '203da7abee4441deaf6f77035f1f415b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.802875 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862f7ab-499e-4d12-8245-ca2f3901624c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:12.809318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 797/1591] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:12 2022] DELETE /identity/v3/users/203da7abee4441deaf6f77035f1f415b => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:12.810390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:12.811272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.811742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.812231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] PATH_INFO: `/v3/projects/291fa40fd4174c9cba616089489f6fd9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.813582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:12.813954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=291fa40fd4174c9cba616089489f6fd9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.814941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:12.816537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.853374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:12.894224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631c5d-48d1-468e-bbb1-a09e61abb1f4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:12.899509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:12.900158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:12.901102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:12.901240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:12.901936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.903849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.932061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291fa40fd4174c9cba616089489f6fd9', 'request_id': 'req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.938646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291fa40fd4174c9cba616089489f6fd9', 'request_id': 'req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:12.946223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291fa40fd4174c9cba616089489f6fd9', 'request_id': 'req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.035947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] The token cache is being invalidate because project 291fa40fd4174c9cba616089489f6fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:13.036519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 798/1592] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:12 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:13.036865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.037631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.038110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.039657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 795/1593] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:12 2022] DELETE /identity/v3/projects/291fa40fd4174c9cba616089489f6fd9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:13.041936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.044556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.046183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.054922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f52fbf-bf63-46e8-9ea8-c913a503ad2a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.127250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e7bef-6fba-4113-8a26-30d0d1e01bbf admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.132058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.132746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.133027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.133305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] PATH_INFO: `/v3/projects/f87286f4fab94c6192ee325fe6c75315` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.136289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87286f4fab94c6192ee325fe6c75315)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.136953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.139778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.140365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.140691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.140961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.141593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.143186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.155285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 799/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:13.156120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.158637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.170835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903d6a1-9026-4c0b-b1de-3177b4751b99 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.175653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badf1f8-9c45-43d9-8aa1-3ffcf028f90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.176271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badf1f8-9c45-43d9-8aa1-3ffcf028f90f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.176547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badf1f8-9c45-43d9-8aa1-3ffcf028f90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.176817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badf1f8-9c45-43d9-8aa1-3ffcf028f90f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.177437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badf1f8-9c45-43d9-8aa1-3ffcf028f90f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.179580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badf1f8-9c45-43d9-8aa1-3ffcf028f90f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.295828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 800/1595] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:13 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:13.295828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.295828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.295828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.305752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87286f4fab94c6192ee325fe6c75315', 'request_id': 'req-6330ef38-cecc-4127-aff3-fadc341268be'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.312750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87286f4fab94c6192ee325fe6c75315', 'request_id': 'req-6330ef38-cecc-4127-aff3-fadc341268be'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.316994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:13.322888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.323028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87286f4fab94c6192ee325fe6c75315', 'request_id': 'req-6330ef38-cecc-4127-aff3-fadc341268be'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.379387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 801/1596] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:13.380484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.381788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.394906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c89c3-6f17-418a-a2bb-8bb8378eb818 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.399541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe5e88-f7cf-4be8-a37c-9c7a6e4c3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.400227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe5e88-f7cf-4be8-a37c-9c7a6e4c3947 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.400508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe5e88-f7cf-4be8-a37c-9c7a6e4c3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.400833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe5e88-f7cf-4be8-a37c-9c7a6e4c3947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.401483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe5e88-f7cf-4be8-a37c-9c7a6e4c3947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.403116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe5e88-f7cf-4be8-a37c-9c7a6e4c3947 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.413657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 802/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:13.415033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae6dd7-eb67-41b2-84d6-b4baa29c7a97 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.415325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae6dd7-eb67-41b2-84d6-b4baa29c7a97 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.415596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae6dd7-eb67-41b2-84d6-b4baa29c7a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.436494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d0ae6dd7-eb67-41b2-84d6-b4baa29c7a97 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:13.437719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d0ae6dd7-eb67-41b2-84d6-b4baa29c7a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.445617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 803/1598] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:13.446540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae6dd7-eb67-41b2-84d6-b4baa29c7a97 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.457351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae6dd7-eb67-41b2-84d6-b4baa29c7a97 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.462320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.462977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.463374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.463722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.464415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.466022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.476764 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] The token cache is being invalidate because project f87286f4fab94c6192ee325fe6c75315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:13.477132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.478371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:13.478558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 804/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:13.479459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.482409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.484133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 796/1600] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:13 2022] DELETE /identity/v3/projects/f87286f4fab94c6192ee325fe6c75315 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:13.484646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.486518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.501179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83581576-bc7b-4fb6-a016-2e39a953f438 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.506357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.506965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.507242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.507985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.508191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.510375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.521737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 805/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:13.522456 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.524139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.536687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe966eb-a2f1-46aa-a1c1-4ac36857d9fa admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.542362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946739f7-7534-42bc-a164-5bc3d5ce53ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.543268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946739f7-7534-42bc-a164-5bc3d5ce53ed admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.543597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946739f7-7534-42bc-a164-5bc3d5ce53ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.543924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946739f7-7534-42bc-a164-5bc3d5ce53ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.544634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946739f7-7534-42bc-a164-5bc3d5ce53ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.546383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946739f7-7534-42bc-a164-5bc3d5ce53ed admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.557650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ef38-cecc-4127-aff3-fadc341268be admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.561679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 806/1602] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:13.561679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.561679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.561679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.565703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a588fc9-8bc0-494c-9c7d-a9ba86c21ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.566752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a588fc9-8bc0-494c-9c7d-a9ba86c21ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.566874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a588fc9-8bc0-494c-9c7d-a9ba86c21ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.567177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a588fc9-8bc0-494c-9c7d-a9ba86c21ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.569256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5a588fc9-8bc0-494c-9c7d-a9ba86c21ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.580974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:13.582299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.582757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a588fc9-8bc0-494c-9c7d-a9ba86c21ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.585013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a588fc9-8bc0-494c-9c7d-a9ba86c21ecd service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.591478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 807/1603] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:13.592415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.593706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.599199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 797/1604] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:13.606060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8098e-fb6c-42a6-b791-857c47b55c29 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.608486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df5fa-4e75-421a-99fa-80d2eaa8521a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.608773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df5fa-4e75-421a-99fa-80d2eaa8521a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.609045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df5fa-4e75-421a-99fa-80d2eaa8521a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.611959 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.613097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.613656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.614499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] PATH_INFO: `/v3/projects/b6da2d12d8b645e897cb5c697b596401/users/9b293b2d6fff400abc4d51b06fa7ec7e/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.630607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-680df5fa-4e75-421a-99fa-80d2eaa8521a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:13.631918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-680df5fa-4e75-421a-99fa-80d2eaa8521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.639250 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6da2d12d8b645e897cb5c697b596401, user_id=9b293b2d6fff400abc4d51b06fa7ec7e, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.643261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 798/1605] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:13.644257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df5fa-4e75-421a-99fa-80d2eaa8521a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.653419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.655697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df5fa-4e75-421a-99fa-80d2eaa8521a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.661163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.661832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.662115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.662386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.663035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.665073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.675997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 799/1606] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:13.676912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.687726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c9f7e-09d8-478c-97eb-bb2f6360fbda admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.692839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82565aa-6a99-4313-bf79-f79824a4d4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.693505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82565aa-6a99-4313-bf79-f79824a4d4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.693779 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82565aa-6a99-4313-bf79-f79824a4d4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.694052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82565aa-6a99-4313-bf79-f79824a4d4d1 admin admin] PATH_INFO: `/v3/projects/2c30c366f2174f808357714dbdc9f74b/users/ffed044a93b341509cebd756933e2f99/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.709868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82565aa-6a99-4313-bf79-f79824a4d4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c30c366f2174f808357714dbdc9f74b, user_id=ffed044a93b341509cebd756933e2f99, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.724523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82565aa-6a99-4313-bf79-f79824a4d4d1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.780097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 808/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:13 2022] PUT /identity/v3/projects/b6da2d12d8b645e897cb5c697b596401/users/9b293b2d6fff400abc4d51b06fa7ec7e/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:13.782479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.842708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224b970-2e64-4d42-b6f0-290958023686 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.847358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.848002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.848282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.848555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.849194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:13.849496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:13.851089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:13.861271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 800/1608] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:13 2022] PUT /identity/v3/projects/2c30c366f2174f808357714dbdc9f74b/users/ffed044a93b341509cebd756933e2f99/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:13.862736 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232644a0-8c8b-4d9f-bdd9-e2d601706cd7 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.862941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232644a0-8c8b-4d9f-bdd9-e2d601706cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.863192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232644a0-8c8b-4d9f-bdd9-e2d601706cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.884683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 809/1609] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:13.884683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:13.890722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-232644a0-8c8b-4d9f-bdd9-e2d601706cd7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:13.892172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-232644a0-8c8b-4d9f-bdd9-e2d601706cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:13.899093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f445139-9f53-4b89-abe0-b0bca6b3ca97 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:13.977438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:13.983706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 801/1610] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:13.985304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b675f9-7e7b-403b-9d73-65d5133e0e69 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.986097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b675f9-7e7b-403b-9d73-65d5133e0e69 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.988809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b675f9-7e7b-403b-9d73-65d5133e0e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:13.988929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55943386-ec07-4c9d-8ed4-e6f25dea6551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:13.989552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55943386-ec07-4c9d-8ed4-e6f25dea6551 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:13.989835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55943386-ec07-4c9d-8ed4-e6f25dea6551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:13.990116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55943386-ec07-4c9d-8ed4-e6f25dea6551 admin admin] PATH_INFO: `/v3/projects/ded503bdd34b498da05cc322ac7f84a0/users/abc2334096894d3ca9c7433e4d8f9cd2/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.024680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55943386-ec07-4c9d-8ed4-e6f25dea6551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded503bdd34b498da05cc322ac7f84a0, user_id=abc2334096894d3ca9c7433e4d8f9cd2, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:14.027447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-10b675f9-7e7b-403b-9d73-65d5133e0e69 None None] MFA Rules not processed for user `9b293b2d6fff400abc4d51b06fa7ec7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:14.028452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55943386-ec07-4c9d-8ed4-e6f25dea6551 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:14.030105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-10b675f9-7e7b-403b-9d73-65d5133e0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.089279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:14.118248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 810/1611] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:13 2022] PUT /identity/v3/projects/ded503bdd34b498da05cc322ac7f84a0/users/abc2334096894d3ca9c7433e4d8f9cd2/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:14.128472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 802/1612] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:14.129334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b675f9-7e7b-403b-9d73-65d5133e0e69 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:14.143546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123107b-2b09-43d7-a52b-ba723ac7c385 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.147143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123107b-2b09-43d7-a52b-ba723ac7c385 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.147563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123107b-2b09-43d7-a52b-ba723ac7c385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.149362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b675f9-7e7b-403b-9d73-65d5133e0e69 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:14.203590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5123107b-2b09-43d7-a52b-ba723ac7c385 None None] MFA Rules not processed for user `ffed044a93b341509cebd756933e2f99`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:14.204851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5123107b-2b09-43d7-a52b-ba723ac7c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.242739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83492780-8a48-4974-b497-f4d809123f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:14.244801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83492780-8a48-4974-b497-f4d809123f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.244801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83492780-8a48-4974-b497-f4d809123f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.245521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83492780-8a48-4974-b497-f4d809123f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.246522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83492780-8a48-4974-b497-f4d809123f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:14.249350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83492780-8a48-4974-b497-f4d809123f44 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:14.277094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 811/1613] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:14.277964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123107b-2b09-43d7-a52b-ba723ac7c385 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:14.289896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123107b-2b09-43d7-a52b-ba723ac7c385 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:14.295041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:14.295645 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.295969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.296224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.296864 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:14.297164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:14.298818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:14.311034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 812/1614] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:14.311914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:14.314048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.326869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bab07-4cf5-4457-a473-b89f1b4eabf8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:14.331425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:14.332191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.332467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.332736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] PATH_INFO: `/v3/projects/94f16b553c95469ca3d7eca676ecf98b` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.333829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f16b553c95469ca3d7eca676ecf98b)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:14.335709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:14.386458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 803/1615] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:14 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:14.387850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274de6e-6703-4189-baec-52ebf3f3a8cf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.388141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274de6e-6703-4189-baec-52ebf3f3a8cf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.388417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274de6e-6703-4189-baec-52ebf3f3a8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.415460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-6274de6e-6703-4189-baec-52ebf3f3a8cf None None] MFA Rules not processed for user `abc2334096894d3ca9c7433e4d8f9cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:14.417269 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6274de6e-6703-4189-baec-52ebf3f3a8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.481311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 804/1616] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:14.482388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274de6e-6703-4189-baec-52ebf3f3a8cf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:14.525211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f16b553c95469ca3d7eca676ecf98b', 'request_id': 'req-fab2abec-4b44-4e32-981b-1fcbb5902728'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:14.531686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f16b553c95469ca3d7eca676ecf98b', 'request_id': 'req-fab2abec-4b44-4e32-981b-1fcbb5902728'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:14.541756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f16b553c95469ca3d7eca676ecf98b', 'request_id': 'req-fab2abec-4b44-4e32-981b-1fcbb5902728'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:14.584908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274de6e-6703-4189-baec-52ebf3f3a8cf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:14.622413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] The token cache is being invalidate because project 94f16b553c95469ca3d7eca676ecf98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:14.622413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:14.622955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:14.625104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 813/1617] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:14 2022] DELETE /identity/v3/projects/94f16b553c95469ca3d7eca676ecf98b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:14.625893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:14.627462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.678831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:14.679741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.680153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.680434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.681261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:14.684595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:14.724864 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2abec-4b44-4e32-981b-1fcbb5902728 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:14.737849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:14.737849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.737849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.737849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.738198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.803689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:14.805826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:14.816678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 814/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:14.817697 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:14.823598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.839306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba896a01-8a74-4645-90ef-964be0f2e9fd service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:14.843891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:14.844479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.844751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.845019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.846013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:14.848258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:14.994879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 805/1619] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:14 2022] POST /identity/v3/projects => generated 375 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:14.995763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:15.001125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:15.039368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 815/1620] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:14 2022] POST /identity/v3/projects => generated 355 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:15.040303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:15.042063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:15.153637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ceee9-78f9-4831-94cd-052bec4298c5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:15.153637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:15.153637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.153637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.154431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.154431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:15.154431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:15.154652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08417804-205c-4a53-b98e-c93da5b301a8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:15.158497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:15.159923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.160313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.160986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.162007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:15.164746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:15.389858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 806/1621] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:14 2022] POST /identity/v3/users => generated 290 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:15.389858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:15.392471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:15.410181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 816/1622] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:15 2022] POST /identity/v3/projects => generated 373 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:15.411823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:15.413902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:15.422384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cae970-e6c5-41c9-b669-e2fef06e5765 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:15.545968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:15.546319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39689f5-fac9-4894-914c-4e590fe774e3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:15.546628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.547051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.547330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] PATH_INFO: `/v3/users/e0ac381249fe49e0b548fa5700343665` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.548513 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ac381249fe49e0b548fa5700343665)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:15.551159 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:15.562081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:15.562822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.567045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.577685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.578600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:15.583574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:15.723835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 817/1623] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:15 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:15.725008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:15.737742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77623ccd-0b8c-4c18-8833-1234d3d41fef admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:15.748253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:15.749091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.749517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.749937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.750784 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:15.753344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:15.831008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ac381249fe49e0b548fa5700343665'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:15.848366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ac381249fe49e0b548fa5700343665'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:15.849701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ac381249fe49e0b548fa5700343665'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:15.857899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 818/1624] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:15 2022] POST /identity/v3/users => generated 278 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:15.859491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:15.862304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:15.864564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ac381249fe49e0b548fa5700343665'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:15.865994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ac381249fe49e0b548fa5700343665'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:15.880708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269140e7-8219-4095-a36d-e5e409cfcfa3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:15.887264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:15.888103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.888384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.888687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] PATH_INFO: `/v3/users/ffed044a93b341509cebd756933e2f99` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.890360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffed044a93b341509cebd756933e2f99)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:15.892418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:15.980081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 807/1625] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:15 2022] DELETE /identity/v3/users/e0ac381249fe49e0b548fa5700343665 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:15.986307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:15.986307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.058005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d418-c851-4aaf-97bb-23ab0f02c5b2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.062978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.063696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.063999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.064272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.064957 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.066666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.077897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 808/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:16.078598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.080286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.117701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c663cf5-99f2-48fe-bf6e-98094fecb74e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.117701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.117701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.117701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.118399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.118399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.118399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.169259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffed044a93b341509cebd756933e2f99'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.169901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffed044a93b341509cebd756933e2f99'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.248297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffed044a93b341509cebd756933e2f99'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.249084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffed044a93b341509cebd756933e2f99'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.258998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 809/1627] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:16 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:16.260779 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.262885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.266245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffed044a93b341509cebd756933e2f99'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.276487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1fa62-6404-445c-b3ed-9830267b776c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.281343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.281974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.282252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.282531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] PATH_INFO: `/v3/users/0af534bcdd5b44cfb010134ae8b682da` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.283671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af534bcdd5b44cfb010134ae8b682da)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.285639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.292581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 819/1628] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:15 2022] DELETE /identity/v3/users/ffed044a93b341509cebd756933e2f99 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:16.292581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.294048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.383352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09757f72-5181-40c0-be94-5d0a67455b09 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.389595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.394366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.394689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.394950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.396099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.397458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.408928 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 820/1629] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:16.409790 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.413664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.430501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08634ae-3df8-444d-9901-193b8ac29934 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.433479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.434164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.434512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.434856 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.435597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.437220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.450410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 821/1630] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:16.450410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.450812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.466027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539ad28-9067-4907-b08e-6bd3d5becb4a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.471526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.472177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.472455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.472728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.473363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.475013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.487712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 822/1631] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:16.488618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.492452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.513867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd051b9-0427-45d7-be20-974bf2e0403a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.514436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af534bcdd5b44cfb010134ae8b682da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.519636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595b586-d5cb-46fd-839b-dbe1498806ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.522045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b586-d5cb-46fd-839b-dbe1498806ee admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.522045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b586-d5cb-46fd-839b-dbe1498806ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.522045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b586-d5cb-46fd-839b-dbe1498806ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.522045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595b586-d5cb-46fd-839b-dbe1498806ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.523392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595b586-d5cb-46fd-839b-dbe1498806ee admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.539391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 823/1632] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:16.539391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.539391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.539391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.541373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af534bcdd5b44cfb010134ae8b682da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.542451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af534bcdd5b44cfb010134ae8b682da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.563601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:16.565357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.575877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af534bcdd5b44cfb010134ae8b682da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.576791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af534bcdd5b44cfb010134ae8b682da'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:16.583305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 824/1633] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:16.585377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.585377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.607742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7788c-c504-4fb2-a6c9-98ee01629460 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.613809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.614616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.614900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.615201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] PATH_INFO: `/v3/projects/9bd260fc68d343b5a65a8d5f91960567/users/d13c0b6305dc43678e4f9c3199085dde/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.633684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd260fc68d343b5a65a8d5f91960567, user_id=d13c0b6305dc43678e4f9c3199085dde, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.637083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.680895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 810/1634] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:16 2022] DELETE /identity/v3/users/0af534bcdd5b44cfb010134ae8b682da => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:16.680895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.680895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.738116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 825/1635] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:16 2022] PUT /identity/v3/projects/9bd260fc68d343b5a65a8d5f91960567/users/d13c0b6305dc43678e4f9c3199085dde/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:16.738116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.738116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.750889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97cde3-e2ab-435c-890b-c525e47c63f6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.757514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:16.802658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e6935-3a76-4a13-a74a-d579fd4f3b5f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.808171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.808828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.809103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.809400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] PATH_INFO: `/v3/projects/e66700e80d124f92a31eb05c97fa9555/users/afd0ff490b384f56a15a19a3887156dc/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.810336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.810938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.811215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.811485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.812245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.813844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.824206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 811/1636] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:16.824937 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.826950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.827143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66700e80d124f92a31eb05c97fa9555, user_id=afd0ff490b384f56a15a19a3887156dc, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.832869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.841412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8051ef-af3f-4c7f-a40c-0747e0f9f3f4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.848596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.849617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.849898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.850171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] PATH_INFO: `/v3/projects/0286c81abf874ed49fbc13c010241a13/users/b9dcfaa0a5e041a286cbdd0bb9745444/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.873751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0286c81abf874ed49fbc13c010241a13, user_id=b9dcfaa0a5e041a286cbdd0bb9745444, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.873751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.939766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 826/1637] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:16 2022] PUT /identity/v3/projects/e66700e80d124f92a31eb05c97fa9555/users/afd0ff490b384f56a15a19a3887156dc/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:16.939766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.942158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.960090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 812/1638] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:16 2022] PUT /identity/v3/projects/0286c81abf874ed49fbc13c010241a13/users/b9dcfaa0a5e041a286cbdd0bb9745444/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:16.960090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.966630 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.019231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ac89-1cc2-45ab-aeee-2d9dd79d7810 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.026063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4b9db-063e-49f2-99b2-c69be96a75c8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.030068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa238696-0560-45b6-ad58-2212753ebf59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.030780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238696-0560-45b6-ad58-2212753ebf59 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.031132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238696-0560-45b6-ad58-2212753ebf59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.031474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238696-0560-45b6-ad58-2212753ebf59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.077212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:17.093336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.094020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.094257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.094722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] PATH_INFO: `/v3/projects/eaeaaa4c9ca54ce1b2882444b9e55040` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.095863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaeaaa4c9ca54ce1b2882444b9e55040)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.097821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.101995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238696-0560-45b6-ad58-2212753ebf59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.104268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238696-0560-45b6-ad58-2212753ebf59 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.115120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 813/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:17.117351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0452f-e718-4a42-b54a-02fbc9f99fc1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.117541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0452f-e718-4a42-b54a-02fbc9f99fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.117918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0452f-e718-4a42-b54a-02fbc9f99fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.146790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d3d0452f-e718-4a42-b54a-02fbc9f99fc1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:17.148458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d3d0452f-e718-4a42-b54a-02fbc9f99fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.159942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 814/1640] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:17.161371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0452f-e718-4a42-b54a-02fbc9f99fc1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.173511 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0452f-e718-4a42-b54a-02fbc9f99fc1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.182995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.183857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.184146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.184419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.185200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.187171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.203549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 815/1641] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:17.203549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.214043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a270469-224e-47f7-9b46-3f87b8101542 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.219609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315848-a1e5-43a4-905e-c670d58fb2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.220502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315848-a1e5-43a4-905e-c670d58fb2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.220808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315848-a1e5-43a4-905e-c670d58fb2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.221084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315848-a1e5-43a4-905e-c670d58fb2d7 admin admin] PATH_INFO: `/v3/projects/d72a205f40ae47289414abe2c2b4d3ad/users/159d270564504fab86a7cee778a4dfd4/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.238997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315848-a1e5-43a4-905e-c670d58fb2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72a205f40ae47289414abe2c2b4d3ad, user_id=159d270564504fab86a7cee778a4dfd4, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.242202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315848-a1e5-43a4-905e-c670d58fb2d7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.286163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaeaaa4c9ca54ce1b2882444b9e55040', 'request_id': 'req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:17.295974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaeaaa4c9ca54ce1b2882444b9e55040', 'request_id': 'req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:17.302782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaeaaa4c9ca54ce1b2882444b9e55040', 'request_id': 'req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:17.423754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 816/1642] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:17 2022] PUT /identity/v3/projects/d72a205f40ae47289414abe2c2b4d3ad/users/159d270564504fab86a7cee778a4dfd4/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:17.425874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.426310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.426582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.462109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] MFA Rules not processed for user `afd0ff490b384f56a15a19a3887156dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:17.464881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.476998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] The token cache is being invalidate because project eaeaaa4c9ca54ce1b2882444b9e55040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:17.476998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:17.477288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:17.485523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 827/1643] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:16 2022] DELETE /identity/v3/projects/eaeaaa4c9ca54ce1b2882444b9e55040 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:17.486509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.489269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.600251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 817/1644] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:17.601177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.602619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.611485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476a0ce-f6ed-4c93-b639-6c2a3cb21149 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.685825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebceb37-04c0-4d53-8f4b-932ea5f397bf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.689017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.689644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.690138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.690530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.691176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.691809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fef19-bc18-4020-a751-0ad63e7817da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.692699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fef19-bc18-4020-a751-0ad63e7817da service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.693232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fef19-bc18-4020-a751-0ad63e7817da service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.693823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fef19-bc18-4020-a751-0ad63e7817da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.697228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.701102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d10fef19-bc18-4020-a751-0ad63e7817da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.722711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fef19-bc18-4020-a751-0ad63e7817da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.727314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 828/1645] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:17.727458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fef19-bc18-4020-a751-0ad63e7817da service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.727549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.727549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.737147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 818/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:17.738499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.738789 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.739064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.740893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa598f3-24d0-429a-9bec-6a64cb8ca871 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.751173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80171e7d-645f-4f75-a7e5-003537882497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.752808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171e7d-645f-4f75-a7e5-003537882497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.753099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171e7d-645f-4f75-a7e5-003537882497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.753372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171e7d-645f-4f75-a7e5-003537882497 admin admin] PATH_INFO: `/v3/projects/9bd260fc68d343b5a65a8d5f91960567/users/d13c0b6305dc43678e4f9c3199085dde/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.755351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80171e7d-645f-4f75-a7e5-003537882497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd260fc68d343b5a65a8d5f91960567, user_id=d13c0b6305dc43678e4f9c3199085dde, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.760773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80171e7d-645f-4f75-a7e5-003537882497 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.775409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] MFA Rules not processed for user `159d270564504fab86a7cee778a4dfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:17.776234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.779875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 829/1647] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:17 2022] PUT /identity/v3/projects/9bd260fc68d343b5a65a8d5f91960567/users/d13c0b6305dc43678e4f9c3199085dde/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:17.781551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0076-2b73-47b8-8bcf-57fa94c0ad63 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.781842 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0076-2b73-47b8-8bcf-57fa94c0ad63 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.782122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be0076-2b73-47b8-8bcf-57fa94c0ad63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.820921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-20be0076-2b73-47b8-8bcf-57fa94c0ad63 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:17.822838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-20be0076-2b73-47b8-8bcf-57fa94c0ad63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.900119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 819/1648] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:17.905106 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.906435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.914797 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 830/1649] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:17.915682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be0076-2b73-47b8-8bcf-57fa94c0ad63 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.931827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cd5de-2967-4046-8bdb-d70e0455e2c2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.935544 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15699489-029c-449a-917e-ab88c90933cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.936384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699489-029c-449a-917e-ab88c90933cc admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.936477 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be0076-2b73-47b8-8bcf-57fa94c0ad63 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:17.936566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699489-029c-449a-917e-ab88c90933cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.936959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699489-029c-449a-917e-ab88c90933cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.937444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15699489-029c-449a-917e-ab88c90933cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.939639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15699489-029c-449a-917e-ab88c90933cc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.941773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:17.942466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.942888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.943167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] PATH_INFO: `/v3/projects/0286c81abf874ed49fbc13c010241a13/users/b9dcfaa0a5e041a286cbdd0bb9745444/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.945097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0286c81abf874ed49fbc13c010241a13, user_id=b9dcfaa0a5e041a286cbdd0bb9745444, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:17.947904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:17.958695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 831/1650] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:17 2022] PUT /identity/v3/projects/0286c81abf874ed49fbc13c010241a13/users/b9dcfaa0a5e041a286cbdd0bb9745444/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:17.959505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.961161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.027841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee3e6a-c60d-48d1-bcd8-28fb1d53c556 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.035754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.036431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.036707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.049768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] PATH_INFO: `/v3/projects/2c30c366f2174f808357714dbdc9f74b` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.049768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c30c366f2174f808357714dbdc9f74b)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.050867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.068804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 820/1651] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:18.070301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.070943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.075223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.119127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] MFA Rules not processed for user `d13c0b6305dc43678e4f9c3199085dde`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:18.120550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.183461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c30c366f2174f808357714dbdc9f74b', 'request_id': 'req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.189756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c30c366f2174f808357714dbdc9f74b', 'request_id': 'req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.196331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 821/1652] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:18.196485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c30c366f2174f808357714dbdc9f74b', 'request_id': 'req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.201570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.203185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.269025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae47a83-4273-4cb5-8ce0-3528e565b7c2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.324192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.324803 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.325084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.325354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.326007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.327992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.361971 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] The token cache is being invalidate because project 2c30c366f2174f808357714dbdc9f74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:18.362487 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.362879 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7fdb5ed6-6bea-450d-ae0b-859f33f8c5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.364416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 832/1653] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:17 2022] DELETE /identity/v3/projects/2c30c366f2174f808357714dbdc9f74b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:18.365603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b18494-9699-4844-a0e3-d3826e2e3b4d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.365885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b18494-9699-4844-a0e3-d3826e2e3b4d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.366155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b18494-9699-4844-a0e3-d3826e2e3b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.386205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-54b18494-9699-4844-a0e3-d3826e2e3b4d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:18.388465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-54b18494-9699-4844-a0e3-d3826e2e3b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.395194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 833/1654] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:18.396439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.396717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.396996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.431025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] MFA Rules not processed for user `b9dcfaa0a5e041a286cbdd0bb9745444`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:18.432580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.484797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 822/1655] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:18 2022] POST /identity/v3/projects => generated 355 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:18.488270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.488270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.502184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 834/1656] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:18.503082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.504653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.518004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f44bd0-8ecf-41ad-b3c3-fe93e387bd2c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.560672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb600cf9-3c59-4a2c-88a0-2e1dc9223326 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.566884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:18.568752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.570058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.570337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.570753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] PATH_INFO: `/v3/users/eca696f5dfaa466fa06782c82ea42758` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.571936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca696f5dfaa466fa06782c82ea42758)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.575234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.579485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.580412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.580692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.580965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.581600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.583526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.695370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 835/1657] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:18 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:18.695621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.697382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.726106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:18.756999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee63797-8a97-47b5-ba1c-1b4f017350f5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.761192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e6b65-2fc8-40a8-8709-870db2a3c18f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.761793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6b65-2fc8-40a8-8709-870db2a3c18f service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.762073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6b65-2fc8-40a8-8709-870db2a3c18f service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.771511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6b65-2fc8-40a8-8709-870db2a3c18f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.771511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-730e6b65-2fc8-40a8-8709-870db2a3c18f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.822420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e6b65-2fc8-40a8-8709-870db2a3c18f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.823892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e6b65-2fc8-40a8-8709-870db2a3c18f service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.832660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 836/1658] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:18.833870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d9d08-613d-4403-8491-61c3e1db8ade None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.834149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d9d08-613d-4403-8491-61c3e1db8ade None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.834419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d9d08-613d-4403-8491-61c3e1db8ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.855706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7d3d9d08-613d-4403-8491-61c3e1db8ade None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:18.856946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7d3d9d08-613d-4403-8491-61c3e1db8ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.865585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 837/1659] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:18.866190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d9d08-613d-4403-8491-61c3e1db8ade None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.875129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca696f5dfaa466fa06782c82ea42758'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.877354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d9d08-613d-4403-8491-61c3e1db8ade None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.882141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.882743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.883117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.883391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.884070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:18.884371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.886056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.889354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca696f5dfaa466fa06782c82ea42758'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.890292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca696f5dfaa466fa06782c82ea42758'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.898174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca696f5dfaa466fa06782c82ea42758'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.898851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca696f5dfaa466fa06782c82ea42758'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:18.903380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 838/1660] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:18.903380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.904889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.918356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25cd4c-f2d3-41c2-814f-04fc7192b697 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.923139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.923767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.924073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.924344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] PATH_INFO: `/v3/projects/d03d408d29084dcb99cf7558556f9df6` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.925452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03d408d29084dcb99cf7558556f9df6)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.927439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.973515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 823/1661] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:18 2022] DELETE /identity/v3/users/eca696f5dfaa466fa06782c82ea42758 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:18.974320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.975659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.045379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0602bfe-6431-4ae1-a7e3-1a5d663c44d3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.050547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.051164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.051460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.051722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.052401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.054337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.084467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03d408d29084dcb99cf7558556f9df6', 'request_id': 'req-b1b28490-8669-47a9-a795-e67bae9fba81'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.090047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03d408d29084dcb99cf7558556f9df6', 'request_id': 'req-b1b28490-8669-47a9-a795-e67bae9fba81'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.097481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03d408d29084dcb99cf7558556f9df6', 'request_id': 'req-b1b28490-8669-47a9-a795-e67bae9fba81'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.163954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 824/1662] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:18 2022] POST /identity/v3/users => generated 284 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:19.163954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.163954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.205315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] The token cache is being invalidate because project d03d408d29084dcb99cf7558556f9df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:19.206063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.206826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.209147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 839/1663] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:18 2022] DELETE /identity/v3/projects/d03d408d29084dcb99cf7558556f9df6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:19.213459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.214926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.226961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c28a9-8cfd-42b9-9aa2-db1a67688095 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.281477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.282104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.282405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.282692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.283586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b28490-8669-47a9-a795-e67bae9fba81 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.285127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.289091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccfe71-e2ff-42c7-94a2-34837aebbabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.289698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccfe71-e2ff-42c7-94a2-34837aebbabf admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.289980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccfe71-e2ff-42c7-94a2-34837aebbabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.290252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccfe71-e2ff-42c7-94a2-34837aebbabf admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.292158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccfe71-e2ff-42c7-94a2-34837aebbabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.294122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccfe71-e2ff-42c7-94a2-34837aebbabf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.305599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 840/1664] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:19.307761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.308070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.308469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.332952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:19.334970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.345257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 841/1665] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:19.346062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.347306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.359945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791175a-172b-4a22-ac44-0c78d993d5fb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.361557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.363040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.365214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.365819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.366095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.366365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.366990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.369656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.373532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 825/1666] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:19.374338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.375650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.391283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c0dc0-4dad-48ac-be2e-d7fb39071d6c service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.395893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0ef8d-cb83-4d0b-8ea1-9359b4072479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.396489 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0ef8d-cb83-4d0b-8ea1-9359b4072479 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.396763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0ef8d-cb83-4d0b-8ea1-9359b4072479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.397029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0ef8d-cb83-4d0b-8ea1-9359b4072479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.397655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0ef8d-cb83-4d0b-8ea1-9359b4072479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.399236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0ef8d-cb83-4d0b-8ea1-9359b4072479 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.410005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 826/1667] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:19.411143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.411426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.411688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.432092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:19.433291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.441004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 827/1668] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:19.441756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.442959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.454668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf2d99-1076-4d8a-8a71-c84e43bbc107 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.459294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.459930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.460207 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.462920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] PATH_INFO: `/v3/projects/0485207d86e243cf8e0edf419b5dfdb6` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.462920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0485207d86e243cf8e0edf419b5dfdb6)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.464606 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.491667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 842/1669] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:19 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:19.492015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.556691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee219e0-f054-4849-a20f-52d8b4f1e0a5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.562649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.563327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.563698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.564200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] PATH_INFO: `/v3/projects/d09c870eef314de1b233c385046faf3e/users/2ef4d822da75492facd22a9939926e9d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.581077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09c870eef314de1b233c385046faf3e, user_id=2ef4d822da75492facd22a9939926e9d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.584047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.602551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0485207d86e243cf8e0edf419b5dfdb6', 'request_id': 'req-4762e896-55b6-459c-a86f-3e81a4ba5ad7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.611734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0485207d86e243cf8e0edf419b5dfdb6', 'request_id': 'req-4762e896-55b6-459c-a86f-3e81a4ba5ad7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.615674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0485207d86e243cf8e0edf419b5dfdb6', 'request_id': 'req-4762e896-55b6-459c-a86f-3e81a4ba5ad7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.699944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] The token cache is being invalidate because project 0485207d86e243cf8e0edf419b5dfdb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:19.700240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.703352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 843/1670] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:19 2022] PUT /identity/v3/projects/d09c870eef314de1b233c385046faf3e/users/2ef4d822da75492facd22a9939926e9d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:19.703477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:19.703555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.703625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 828/1671] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:19 2022] DELETE /identity/v3/projects/0485207d86e243cf8e0edf419b5dfdb6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:19.703625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.704815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.707518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.746768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:19.780698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225903-1eca-4541-b42b-a7c4a4dd2ef7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.781412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762e896-55b6-459c-a86f-3e81a4ba5ad7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.785376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8a3b9-3657-4e28-b8ff-2864731ca383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.785992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8a3b9-3657-4e28-b8ff-2864731ca383 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.786238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.786361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8a3b9-3657-4e28-b8ff-2864731ca383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.786526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8a3b9-3657-4e28-b8ff-2864731ca383 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.786876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.787158 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.787257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8a3b9-3657-4e28-b8ff-2864731ca383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:19.787432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] PATH_INFO: `/v3/projects/1b07ff4612e7420d84fe8d0b75e04c6b/users/873a9e099149474690c4872149c5177e/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.787520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8a3b9-3657-4e28-b8ff-2864731ca383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.789135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8a3b9-3657-4e28-b8ff-2864731ca383 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.800831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 844/1672] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:19.802200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.802429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.802710 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.803293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b07ff4612e7420d84fe8d0b75e04c6b, user_id=873a9e099149474690c4872149c5177e, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.806332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.823513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:19.824739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.832625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 845/1673] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:19.833362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.834538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.846749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91014e42-26b1-4e79-892c-78fb0db8d226 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.851522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.852312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.852588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.852858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.853594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:19.855573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:19.865915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:19.882508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 829/1674] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:19 2022] PUT /identity/v3/projects/1b07ff4612e7420d84fe8d0b75e04c6b/users/873a9e099149474690c4872149c5177e/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:19.883569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.885842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.941177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58731c2-24f4-4831-bd21-8bc7f6f4ed78 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:19.945121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e812-a994-468b-af0a-1ec665cc6b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:19.945581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 846/1675] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:19 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:19.945722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e812-a994-468b-af0a-1ec665cc6b58 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:19.946001 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e812-a994-468b-af0a-1ec665cc6b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:19.946348 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e812-a994-468b-af0a-1ec665cc6b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:19.946446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:19.948893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6935e812-a994-468b-af0a-1ec665cc6b58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.949081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:19.961217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd4f3e-e86f-44ea-b61b-3d5a5bfe50ec admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.008445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e812-a994-468b-af0a-1ec665cc6b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.010372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e812-a994-468b-af0a-1ec665cc6b58 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.018248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.018863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.019142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.019433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.019537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 830/1676] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:20.020182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.020717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85ec5-26a3-4aa6-8f63-2cd73335e051 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.021034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85ec5-26a3-4aa6-8f63-2cd73335e051 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.021276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85ec5-26a3-4aa6-8f63-2cd73335e051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.021844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.033773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 847/1677] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:20.034618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.047001 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7ea85ec5-26a3-4aa6-8f63-2cd73335e051 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:20.047235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de38ce-1536-41f6-8570-901af91001f9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.048299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7ea85ec5-26a3-4aa6-8f63-2cd73335e051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:20.052007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.052617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.052901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.053179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.053816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.055712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.058545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 831/1678] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:20.059588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea85ec5-26a3-4aa6-8f63-2cd73335e051 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.072860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea85ec5-26a3-4aa6-8f63-2cd73335e051 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.079364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.080009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.080294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.080567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.081218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:20.081528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.083143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.099034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 832/1679] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:20.099034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.107536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de0b14-a2cb-4ce5-ad58-4829679d0fb7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.112287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.112878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.113177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.113510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.114181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.115817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.127136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 833/1680] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:20.127975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.138234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985d4fe-74e9-4aba-a5cc-d3fe49d4e851 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.142845 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.143512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.143780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.144080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.144699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.146358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.156259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 834/1681] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:20.157010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.166738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a496-2b17-44d8-a142-5c90214a0193 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.171480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ed4f5-080b-4051-9d65-e106a8bda984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.172144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ed4f5-080b-4051-9d65-e106a8bda984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.172425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ed4f5-080b-4051-9d65-e106a8bda984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.172702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ed4f5-080b-4051-9d65-e106a8bda984 admin admin] PATH_INFO: `/v3/projects/d09c870eef314de1b233c385046faf3e/users/2ef4d822da75492facd22a9939926e9d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.174618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ed4f5-080b-4051-9d65-e106a8bda984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09c870eef314de1b233c385046faf3e, user_id=2ef4d822da75492facd22a9939926e9d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.177268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ed4f5-080b-4051-9d65-e106a8bda984 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.185478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:20.199743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 835/1682] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:20 2022] PUT /identity/v3/projects/d09c870eef314de1b233c385046faf3e/users/2ef4d822da75492facd22a9939926e9d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:20.199743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038e55d-469b-4b03-9787-e12f7e75ed79 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.199743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038e55d-469b-4b03-9787-e12f7e75ed79 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.199743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038e55d-469b-4b03-9787-e12f7e75ed79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.200166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 848/1683] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:20.200166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.215034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d038e55d-469b-4b03-9787-e12f7e75ed79 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:20.216290 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d038e55d-469b-4b03-9787-e12f7e75ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:20.260714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83a26-0d9a-485c-aa82-ce3943ecd2fa admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.265525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d66c4f-ecc6-4af4-8c78-07b6ee888aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.266130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d66c4f-ecc6-4af4-8c78-07b6ee888aee admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.266409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d66c4f-ecc6-4af4-8c78-07b6ee888aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.266676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d66c4f-ecc6-4af4-8c78-07b6ee888aee admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.267298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d66c4f-ecc6-4af4-8c78-07b6ee888aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.269214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d66c4f-ecc6-4af4-8c78-07b6ee888aee admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.276728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 836/1684] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:20.277647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038e55d-469b-4b03-9787-e12f7e75ed79 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.287595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038e55d-469b-4b03-9787-e12f7e75ed79 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.293800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.294554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.294896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.295232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] PATH_INFO: `/v3/projects/1b07ff4612e7420d84fe8d0b75e04c6b/users/873a9e099149474690c4872149c5177e/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.297257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b07ff4612e7420d84fe8d0b75e04c6b, user_id=873a9e099149474690c4872149c5177e, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.300084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.310625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 837/1685] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:20 2022] PUT /identity/v3/projects/1b07ff4612e7420d84fe8d0b75e04c6b/users/873a9e099149474690c4872149c5177e/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:20.311472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.373460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cadf2d-2a76-4a56-815c-9e19d734e4db admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.378379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.378980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.379256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.379525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.380182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.381783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.391993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 838/1686] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:20.392776 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.399079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 849/1687] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/projects => generated 369 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:20.401434 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507f6b-abc9-405e-a31d-d185be5c0da8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.401759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507f6b-abc9-405e-a31d-d185be5c0da8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.402031 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507f6b-abc9-405e-a31d-d185be5c0da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.405661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108a504-200a-412b-8d10-1c053a8c3e8b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.430749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c0507f6b-abc9-405e-a31d-d185be5c0da8 None None] MFA Rules not processed for user `2ef4d822da75492facd22a9939926e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:20.432083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c0507f6b-abc9-405e-a31d-d185be5c0da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:20.463508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.464218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.464512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.464807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.465491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.467420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.501137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 850/1688] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:20.502364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507f6b-abc9-405e-a31d-d185be5c0da8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.512551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507f6b-abc9-405e-a31d-d185be5c0da8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.521453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61a900-7366-4fc6-86e6-da7c22b42a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.522078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61a900-7366-4fc6-86e6-da7c22b42a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.522361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61a900-7366-4fc6-86e6-da7c22b42a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.522640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61a900-7366-4fc6-86e6-da7c22b42a36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.523264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61a900-7366-4fc6-86e6-da7c22b42a36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:20.523619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61a900-7366-4fc6-86e6-da7c22b42a36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.525216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61a900-7366-4fc6-86e6-da7c22b42a36 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.533007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:20.538247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 851/1689] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:20.539592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d0774-97aa-4526-bd3f-baf91664088e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.539930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d0774-97aa-4526-bd3f-baf91664088e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.540210 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d0774-97aa-4526-bd3f-baf91664088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.569471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ae1d0774-97aa-4526-bd3f-baf91664088e None None] MFA Rules not processed for user `873a9e099149474690c4872149c5177e`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:20.570673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ae1d0774-97aa-4526-bd3f-baf91664088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:20.577672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 839/1690] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:20.577672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.586002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fa710-c09a-42a3-b136-990f00cb8697 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.590755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.591406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.591668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.591973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] PATH_INFO: `/v3/projects/4b5e76f54e504414aa904a2760e1b123/users/49818aa981cc4fbc8d6808fbe1065d6f/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.608089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5e76f54e504414aa904a2760e1b123, user_id=49818aa981cc4fbc8d6808fbe1065d6f, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.610855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.638760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 852/1691] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:20.640960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d0774-97aa-4526-bd3f-baf91664088e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.649137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:20.653447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d0774-97aa-4526-bd3f-baf91664088e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.658248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.658860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.659144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.659420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.660104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.662283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.710840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 840/1692] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:20 2022] PUT /identity/v3/projects/4b5e76f54e504414aa904a2760e1b123/users/49818aa981cc4fbc8d6808fbe1065d6f/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:20.711914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.714237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:20.771582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d65261-8eba-45e5-a711-27c1b1e5ab2b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.775841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.776441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.776721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.776991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.811664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 853/1693] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:20.811664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.819649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbb52d-3048-4924-b338-d5015df97865 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.839545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.841072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.851589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 841/1694] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:20.852425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.879743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d64ea1-1fe8-47b9-b854-41a1b7af2dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.880527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d64ea1-1fe8-47b9-b854-41a1b7af2dad admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.880894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d64ea1-1fe8-47b9-b854-41a1b7af2dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.881172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d64ea1-1fe8-47b9-b854-41a1b7af2dad admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.881826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d64ea1-1fe8-47b9-b854-41a1b7af2dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.883831 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d64ea1-1fe8-47b9-b854-41a1b7af2dad admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.917369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb9667-4cea-441f-bb5c-08828182d249 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.922249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.922857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.923131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.923400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.924086 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:20.925970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:20.936956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 842/1695] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:20.937923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:20.947755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11573885-3ec7-4ce6-b5d8-9b4636eec25b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:20.951432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:20.952014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.952286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.952553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.994432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 854/1696] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:20.994432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf015-1a92-4279-87e7-585a9b5cee70 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.994432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf015-1a92-4279-87e7-585a9b5cee70 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.994783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf015-1a92-4279-87e7-585a9b5cee70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.037810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.039645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.049609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3eacf015-1a92-4279-87e7-585a9b5cee70 None None] MFA Rules not processed for user `49818aa981cc4fbc8d6808fbe1065d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:21.050857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3eacf015-1a92-4279-87e7-585a9b5cee70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:21.052076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 843/1697] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:21.052905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.064980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3df92-61c8-44bc-9db0-495583d5f495 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.125465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 855/1698] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:21.126854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacf015-1a92-4279-87e7-585a9b5cee70 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.128380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.129383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.129502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.129696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.130380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.132032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.137969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacf015-1a92-4279-87e7-585a9b5cee70 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.142540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.145666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 844/1699] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:21.145801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.145801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.145801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.146039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.146111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.146111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.157409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 856/1700] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:21.159140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51ad5-20eb-4804-9799-a0229f3106e5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.159349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.163999 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.164678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.165026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.166183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.167304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.169934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.175292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ec63c-780b-40a2-a281-e95ef86dcff6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.180617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.181608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.181989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.182338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.183072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.185089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.343651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 845/1701] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:21 2022] POST /identity/v3/users => generated 302 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:21.343651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.350899 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c64aa-e00f-46df-879c-ec34853b40a9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.355585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.356197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.356471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.356742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.357370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.358952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.369666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 846/1702] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:21.370463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.380183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6efd4e-affd-4f65-beb6-a54fa54ccbee admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.385469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.386076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.388456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.388456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] PATH_INFO: `/v3/projects/91220a5b455a48a9b288514d8e01623e/users/17b82a229cce49ca9d682e844df697a6/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.401406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91220a5b455a48a9b288514d8e01623e, user_id=17b82a229cce49ca9d682e844df697a6, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.404053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.425707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 857/1703] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:21 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:21.425707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.483062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 847/1704] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:21 2022] PUT /identity/v3/projects/91220a5b455a48a9b288514d8e01623e/users/17b82a229cce49ca9d682e844df697a6/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:21.483062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.494526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99e41-8b53-431d-8330-73a2ceb9052d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.557288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b87913-5223-4d95-b2c0-f559e2f94797 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.562436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.563280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.563568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.563873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] PATH_INFO: `/v3/projects/149e2dc638074494ba2d8b02d47f7382/users/21db9e77db0c491090e3b7d9188d8ada/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.567764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.568626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.568871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.569130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.569836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.572493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.593783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 858/1705] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:21.593783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.593953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149e2dc638074494ba2d8b02d47f7382, user_id=21db9e77db0c491090e3b7d9188d8ada, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.595379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.597899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffc2e4-866d-4474-acbb-9b9f76071b67 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.602705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcd26c-f5e4-49ab-9bc3-e2e338c3642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.603419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd26c-f5e4-49ab-9bc3-e2e338c3642d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.603690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd26c-f5e4-49ab-9bc3-e2e338c3642d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.603985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd26c-f5e4-49ab-9bc3-e2e338c3642d admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.604607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcd26c-f5e4-49ab-9bc3-e2e338c3642d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.606536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcd26c-f5e4-49ab-9bc3-e2e338c3642d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.715618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 848/1706] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:21 2022] PUT /identity/v3/projects/149e2dc638074494ba2d8b02d47f7382/users/21db9e77db0c491090e3b7d9188d8ada/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:21.716499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.771621 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841a8b-3186-4187-859a-ef963f0a2513 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.776046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.776698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.777040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.777321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.779087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:21.793212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:21.799383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 859/1707] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:21 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:21.801215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c46796-db99-4d4e-aa25-a009c7361649 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.801598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c46796-db99-4d4e-aa25-a009c7361649 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.801982 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c46796-db99-4d4e-aa25-a009c7361649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.830005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c3c46796-db99-4d4e-aa25-a009c7361649 None None] MFA Rules not processed for user `17b82a229cce49ca9d682e844df697a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:21.831326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c3c46796-db99-4d4e-aa25-a009c7361649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:21.838313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.839806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.848816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 849/1708] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:21.851105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.905618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 860/1709] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:21.905618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66c57-745a-44f3-9831-fcc921363a63 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.905618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66c57-745a-44f3-9831-fcc921363a63 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.905618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66c57-745a-44f3-9831-fcc921363a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.926652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ee959-e0ac-4ef1-ba0d-ed5b67833d3c service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.932127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.932805 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.933089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.933364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.933812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-aae66c57-745a-44f3-9831-fcc921363a63 None None] MFA Rules not processed for user `21db9e77db0c491090e3b7d9188d8ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:21.934008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.935015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aae66c57-745a-44f3-9831-fcc921363a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:21.935634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.946039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 850/1710] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:21.947648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.957080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead912-7ca2-4835-b1ee-b75a1eee0a06 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.961817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.962493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.962773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.963045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.963679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.965305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.985285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 851/1711] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:21.989005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.999243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618132a0-fed5-4f41-b5b9-3b7d8fc0e9aa admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.002387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 861/1712] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:22.003479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae66c57-745a-44f3-9831-fcc921363a63 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.004252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.008124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.008124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.008124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.008124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.008124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.014702 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae66c57-745a-44f3-9831-fcc921363a63 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.019678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.020485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.020799 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.021080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] PATH_INFO: `/v3/projects/d66aec8ea88b4ac5ac107037bc85593d/users/64b0b9c929f8452380ef923190caefad/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.037164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d66aec8ea88b4ac5ac107037bc85593d, user_id=64b0b9c929f8452380ef923190caefad, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.039953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.097357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 852/1713] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:21 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:22.098438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.123456 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 862/1714] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:22 2022] PUT /identity/v3/projects/d66aec8ea88b4ac5ac107037bc85593d/users/64b0b9c929f8452380ef923190caefad/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:22.124616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.164448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6348c-b9c9-4871-9e21-7c24346af6a5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.191494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637080a-d9b4-4f06-81f0-2b0915a4ed7d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.199386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9dc739-2506-47eb-8470-e504c37743a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.200260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9dc739-2506-47eb-8470-e504c37743a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.200621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9dc739-2506-47eb-8470-e504c37743a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.200957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9dc739-2506-47eb-8470-e504c37743a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.201656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9dc739-2506-47eb-8470-e504c37743a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.203562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9dc739-2506-47eb-8470-e504c37743a8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.225611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.226395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.227778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:22.228371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.228709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.229386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.231028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.240896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 853/1715] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:22.241825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.251632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655020-95ca-4da6-bbb5-cfa74fe66c85 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.256226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.256797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.257074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.257341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.257943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.260698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.387276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 863/1716] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:22 2022] POST /identity/v3/projects => generated 369 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:22.388846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62561-6b51-4a0a-b404-4bbe767f16f2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.389131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62561-6b51-4a0a-b404-4bbe767f16f2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.389404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62561-6b51-4a0a-b404-4bbe767f16f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.407945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-cbc62561-6b51-4a0a-b404-4bbe767f16f2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:22.409144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-cbc62561-6b51-4a0a-b404-4bbe767f16f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:22.427663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 854/1717] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:22 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:22.428941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.440809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5068b60-7974-4e64-942c-a313fdfa010d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.487884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 864/1718] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:22.488634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc62561-6b51-4a0a-b404-4bbe767f16f2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.499099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc62561-6b51-4a0a-b404-4bbe767f16f2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.505368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.506180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.506527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.507022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.508252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.514174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.520190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.520865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.521148 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.521423 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] PATH_INFO: `/v3/projects/4d93751d2cb14bdfb172c35acf9d9d69/users/d0177c16b13e4e7fa2b8367e4b476162/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.537835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d93751d2cb14bdfb172c35acf9d9d69, user_id=d0177c16b13e4e7fa2b8367e4b476162, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.541248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.650579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 865/1719] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:22 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:22.651526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.660498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 855/1720] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:22 2022] PUT /identity/v3/projects/4d93751d2cb14bdfb172c35acf9d9d69/users/d0177c16b13e4e7fa2b8367e4b476162/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:22.661210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.664966 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:22.675297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e4df0-230d-41a4-a2b2-51463be32c1b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.746403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b54b6b-b70f-46fe-ba2f-0320252c342e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.748697 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.749360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.749633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.749900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.750304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.750617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.750916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.751200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.751733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.752649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.762264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.763405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 866/1721] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:22.763754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.764277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.765983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:22.778130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 856/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:22.778130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.780706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5caeb-b7aa-4e89-8aed-25c85fe9d512 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.786118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.786592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03844bf4-56bd-4005-b001-c25d5592b35c service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.786752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.787026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.787540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] PATH_INFO: `/v3/users/9b293b2d6fff400abc4d51b06fa7ec7e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.788708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b293b2d6fff400abc4d51b06fa7ec7e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.791469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.793428 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868f6e5-b7bc-4584-87d3-5e6475e35ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.794164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868f6e5-b7bc-4584-87d3-5e6475e35ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.794521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868f6e5-b7bc-4584-87d3-5e6475e35ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.794993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868f6e5-b7bc-4584-87d3-5e6475e35ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.795852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868f6e5-b7bc-4584-87d3-5e6475e35ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.798128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868f6e5-b7bc-4584-87d3-5e6475e35ddb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.812572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 857/1723] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:22.812572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d82b-26f9-4348-9180-a41559412200 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.812572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d82b-26f9-4348-9180-a41559412200 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.812572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d82b-26f9-4348-9180-a41559412200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.841452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3f64d82b-26f9-4348-9180-a41559412200 None None] MFA Rules not processed for user `d0177c16b13e4e7fa2b8367e4b476162`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:22.842654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3f64d82b-26f9-4348-9180-a41559412200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:22.920104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 858/1724] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:22.921014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64d82b-26f9-4348-9180-a41559412200 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:22.932104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64d82b-26f9-4348-9180-a41559412200 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:22.936953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:22.939926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:22.939926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:22.939926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] PATH_INFO: `/v3/projects/8ae1668b65d0401e9e057f3c9d0252bc/users/3af3a7e00814427696255707d9ed80f8/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:22.955309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae1668b65d0401e9e057f3c9d0252bc, user_id=3af3a7e00814427696255707d9ed80f8, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:22.958218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:22.999851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b293b2d6fff400abc4d51b06fa7ec7e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.000566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b293b2d6fff400abc4d51b06fa7ec7e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.043340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 859/1725] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:22 2022] PUT /identity/v3/projects/8ae1668b65d0401e9e057f3c9d0252bc/users/3af3a7e00814427696255707d9ed80f8/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:23.046304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:23.065467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b293b2d6fff400abc4d51b06fa7ec7e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.066100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b293b2d6fff400abc4d51b06fa7ec7e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.075487 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b293b2d6fff400abc4d51b06fa7ec7e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.097387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 867/1726] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:22 2022] DELETE /identity/v3/users/9b293b2d6fff400abc4d51b06fa7ec7e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:23.098465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:23.100302 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.112444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52abd-3a4c-4b8a-8545-a8dd74fbb293 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:23.179659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:23.180695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.181353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.181889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] PATH_INFO: `/v3/projects/3e49d64964d643bf87fddbdaaf5cf0e1/users/b913ae43cd9e431fbc2dab77e91ce902/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.189272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b745b2b-e4b3-4396-9b54-2422d2c7082c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:23.194538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:23.195245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.195530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.195857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] PATH_INFO: `/v3/users/afd0ff490b384f56a15a19a3887156dc` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.204399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd0ff490b384f56a15a19a3887156dc)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:23.204529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e49d64964d643bf87fddbdaaf5cf0e1, user_id=b913ae43cd9e431fbc2dab77e91ce902, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:23.204623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:23.204707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:23.380229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 860/1727] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:23 2022] PUT /identity/v3/projects/3e49d64964d643bf87fddbdaaf5cf0e1/users/b913ae43cd9e431fbc2dab77e91ce902/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:23.381119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:23.382747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.451453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd47a-664b-4835-9314-c8df0428f27f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:23.458230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:23.459675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.460137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.460559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.461813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:23.464693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:23.536554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd0ff490b384f56a15a19a3887156dc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.537233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd0ff490b384f56a15a19a3887156dc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.622902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 861/1728] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:23 2022] POST /identity/v3/projects => generated 383 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:23.628146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:23.630703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.664269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd0ff490b384f56a15a19a3887156dc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.665301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd0ff490b384f56a15a19a3887156dc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.690269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-57be0b76-9655-4c21-bc64-7d8b70e75369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd0ff490b384f56a15a19a3887156dc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:23.745747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 868/1729] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:23 2022] DELETE /identity/v3/users/afd0ff490b384f56a15a19a3887156dc => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:23.751141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0229-e57a-469e-a303-769d333a423d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.751898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0229-e57a-469e-a303-769d333a423d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.752414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0229-e57a-469e-a303-769d333a423d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.773894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e40400-0678-4b0e-8005-19a088d73438 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:23.777187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d32a0229-e57a-469e-a303-769d333a423d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:23.778436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d32a0229-e57a-469e-a303-769d333a423d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.849890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:23.850505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.850784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.851052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.851858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:23.855926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 869/1730] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:23.856097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:23.856178 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0229-e57a-469e-a303-769d333a423d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:23.862439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d32a0229-e57a-469e-a303-769d333a423d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.867291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 862/1731] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:23.872202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:23.874030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.875934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0229-e57a-469e-a303-769d333a423d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:23.882958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aeab6-a8a1-4cb4-ac92-37b1f57e6750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:23.884618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aeab6-a8a1-4cb4-ac92-37b1f57e6750 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.884908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aeab6-a8a1-4cb4-ac92-37b1f57e6750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.885186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aeab6-a8a1-4cb4-ac92-37b1f57e6750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.885987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aeab6-a8a1-4cb4-ac92-37b1f57e6750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:23.888393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aeab6-a8a1-4cb4-ac92-37b1f57e6750 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:23.897457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8baa-3e6a-409b-b07e-66e5163200b3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:23.899190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 870/1732] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:23.900696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.900929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.901222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.906563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:23.907255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.907612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.907988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.908979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:23.910993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:23.920409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:23.921627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.930446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 871/1733] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:23.931573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:23.932847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:23.988811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5046e3-80e0-4552-be7f-2f1587df7cdf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:23.992439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:23.993045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.993839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.994113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.006852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 863/1734] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:23 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:24.006852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.008465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.008585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.010981 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.021584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 872/1735] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:24.021824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37347b08-48c0-4248-933f-1ef7736f90af admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.022342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.026984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.027741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.028013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.028243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] PATH_INFO: `/v3/projects/8ae1668b65d0401e9e057f3c9d0252bc/users/3af3a7e00814427696255707d9ed80f8/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.030255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae1668b65d0401e9e057f3c9d0252bc, user_id=3af3a7e00814427696255707d9ed80f8, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.035893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebaed8-1071-4541-ba00-d84072ab430b service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.041044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.041855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.042254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.042600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] PATH_INFO: `/v3/projects/3e49d64964d643bf87fddbdaaf5cf0e1/users/b913ae43cd9e431fbc2dab77e91ce902/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.045960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e49d64964d643bf87fddbdaaf5cf0e1, user_id=b913ae43cd9e431fbc2dab77e91ce902, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.047474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.049219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.061228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 864/1736] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:24 2022] PUT /identity/v3/projects/8ae1668b65d0401e9e057f3c9d0252bc/users/3af3a7e00814427696255707d9ed80f8/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:24.062080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.065025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 873/1737] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:24 2022] PUT /identity/v3/projects/3e49d64964d643bf87fddbdaaf5cf0e1/users/b913ae43cd9e431fbc2dab77e91ce902/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:24.066394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.106338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:24.149030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4c0c1-cd1c-4345-bd9e-9ddc78744d4a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.179997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adfeae-27da-4cdb-9b14-9ef7ea4e4bba admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.188160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515b962-f27f-48bc-90ff-2f1c23be82ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.189031 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515b962-f27f-48bc-90ff-2f1c23be82ee admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.189483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515b962-f27f-48bc-90ff-2f1c23be82ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.189807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515b962-f27f-48bc-90ff-2f1c23be82ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.190584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515b962-f27f-48bc-90ff-2f1c23be82ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.193628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515b962-f27f-48bc-90ff-2f1c23be82ee admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.207834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 874/1738] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:24.207834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233843f-1fe4-4ca5-9fe9-64a9fe49b43a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.207834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233843f-1fe4-4ca5-9fe9-64a9fe49b43a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.207834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233843f-1fe4-4ca5-9fe9-64a9fe49b43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.220953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb6601-7465-48dd-aab0-c7ddfb26d4f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.221573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb6601-7465-48dd-aab0-c7ddfb26d4f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.221857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb6601-7465-48dd-aab0-c7ddfb26d4f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.222130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb6601-7465-48dd-aab0-c7ddfb26d4f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.231935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb6601-7465-48dd-aab0-c7ddfb26d4f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.233368 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb6601-7465-48dd-aab0-c7ddfb26d4f7 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.236048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0233843f-1fe4-4ca5-9fe9-64a9fe49b43a None None] MFA Rules not processed for user `3af3a7e00814427696255707d9ed80f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:24.237251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0233843f-1fe4-4ca5-9fe9-64a9fe49b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.243400 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 865/1739] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:24.245034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.245364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.245643 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.277836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] MFA Rules not processed for user `b913ae43cd9e431fbc2dab77e91ce902`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:24.279482 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.326707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 875/1740] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:24.327591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233843f-1fe4-4ca5-9fe9-64a9fe49b43a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.348502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233843f-1fe4-4ca5-9fe9-64a9fe49b43a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.355350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.355977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.356337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.356777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] PATH_INFO: `/v3/projects/f9f46f2db2c94c949ec576323f25fe5a/users/0cd7d6c2be8a4852be716a3bc7312885/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.376240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f46f2db2c94c949ec576323f25fe5a, user_id=0cd7d6c2be8a4852be716a3bc7312885, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.379673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.394039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 866/1741] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:24.394915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.396850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.459616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018a9d8-96eb-4ce1-9f90-bda4dae091d5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.463615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.464247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.464526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.464791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.473841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.475380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.487471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 867/1742] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:24.495030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 876/1743] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:24 2022] PUT /identity/v3/projects/f9f46f2db2c94c949ec576323f25fe5a/users/0cd7d6c2be8a4852be716a3bc7312885/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:24.496923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.518820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.520908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.521847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b1ee6-d776-45d3-aec9-0269307fb0e2 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.599996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.601315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.601837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.602335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.604223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.607049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.609726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:24.609851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a941426-cd6a-4c71-af9a-e1eaa0c9b550 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.618207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.619625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.620187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.620472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.621251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 868/1744] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:24.632612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.635602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.655134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dfa90-b122-4a61-a488-229c3b8a8dbc admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.661681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.663596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.663963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.665766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] PATH_INFO: `/v3/projects/b6da2d12d8b645e897cb5c697b596401` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.669170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6da2d12d8b645e897cb5c697b596401)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.671340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.729366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.731243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.747226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 877/1745] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:24.748128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.760744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d923279-cdaa-454b-8fdb-e480ad37fd45 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.765565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988387b-b663-4a99-bc76-ac6ec7872c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.766625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988387b-b663-4a99-bc76-ac6ec7872c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.766906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988387b-b663-4a99-bc76-ac6ec7872c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.767187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988387b-b663-4a99-bc76-ac6ec7872c19 admin admin] PATH_INFO: `/v3/projects/f9f46f2db2c94c949ec576323f25fe5a/users/0cd7d6c2be8a4852be716a3bc7312885/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.769982 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988387b-b663-4a99-bc76-ac6ec7872c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f46f2db2c94c949ec576323f25fe5a, user_id=0cd7d6c2be8a4852be716a3bc7312885, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.772696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988387b-b663-4a99-bc76-ac6ec7872c19 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.784165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 878/1746] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:24 2022] PUT /identity/v3/projects/f9f46f2db2c94c949ec576323f25fe5a/users/0cd7d6c2be8a4852be716a3bc7312885/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:24.785564 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a2c34-1a17-4422-b5b7-423c17bfec7f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.788240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a2c34-1a17-4422-b5b7-423c17bfec7f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.788240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a2c34-1a17-4422-b5b7-423c17bfec7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.815497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-878a2c34-1a17-4422-b5b7-423c17bfec7f None None] MFA Rules not processed for user `64b0b9c929f8452380ef923190caefad`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:24.816833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-878a2c34-1a17-4422-b5b7-423c17bfec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.904493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6da2d12d8b645e897cb5c697b596401', 'request_id': 'req-23ded971-57a1-475a-b172-7bb974716442'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:24.908218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 879/1747] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:24.909681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701fa1e-2428-44c2-8dfe-d4d6076075c7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.909861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701fa1e-2428-44c2-8dfe-d4d6076075c7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.910136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701fa1e-2428-44c2-8dfe-d4d6076075c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.914323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6da2d12d8b645e897cb5c697b596401', 'request_id': 'req-23ded971-57a1-475a-b172-7bb974716442'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:24.922162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6da2d12d8b645e897cb5c697b596401', 'request_id': 'req-23ded971-57a1-475a-b172-7bb974716442'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:24.940478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7701fa1e-2428-44c2-8dfe-d4d6076075c7 None None] MFA Rules not processed for user `0cd7d6c2be8a4852be716a3bc7312885`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:24.941771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7701fa1e-2428-44c2-8dfe-d4d6076075c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.001701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] The token cache is being invalidate because project b6da2d12d8b645e897cb5c697b596401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:25.002048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.002699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.004344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 869/1748] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:24 2022] DELETE /identity/v3/projects/b6da2d12d8b645e897cb5c697b596401 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:25.005140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.006471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.011142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 880/1749] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:25.012165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701fa1e-2428-44c2-8dfe-d4d6076075c7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.068802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701fa1e-2428-44c2-8dfe-d4d6076075c7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:25.072601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:25.073214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:25.073495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:25.073768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:25.075582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.080839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ded971-57a1-475a-b172-7bb974716442 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:25.085608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:25.086246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:25.086496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:25.086762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] PATH_INFO: `/v3/projects/e66700e80d124f92a31eb05c97fa9555` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:25.088085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66700e80d124f92a31eb05c97fa9555)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:25.089990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:25.135391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:25.136866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:25.145629 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 881/1750] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:25.146404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.147674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.159017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f415d-0f8c-4a1a-bb34-491604185abe service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:25.163123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:25.163847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:25.164251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:25.164535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:25.166367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.226252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:25.227847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:25.236704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 882/1751] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:25.237462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.238844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.253183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccbe64-1163-406e-9a45-27526d439a28 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:25.258046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:25.258680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:25.259503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:25.259823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:25.260454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:25.262420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:25.334296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66700e80d124f92a31eb05c97fa9555', 'request_id': 'req-c3e2d4af-8a02-471e-9705-1a8ded14fbad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.340041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66700e80d124f92a31eb05c97fa9555', 'request_id': 'req-c3e2d4af-8a02-471e-9705-1a8ded14fbad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.346596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66700e80d124f92a31eb05c97fa9555', 'request_id': 'req-c3e2d4af-8a02-471e-9705-1a8ded14fbad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.465390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 883/1752] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:25 2022] POST /identity/v3/projects => generated 405 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:25.466299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.467863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.529501 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadde41-ab37-493f-a54c-67fd26f5d3a9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:25.534139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:25.534761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:25.535040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:25.535314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] PATH_INFO: `/v3/users/d13c0b6305dc43678e4f9c3199085dde` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:25.536457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13c0b6305dc43678e4f9c3199085dde)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:25.538416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:25.557077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] The token cache is being invalidate because project e66700e80d124f92a31eb05c97fa9555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:25.557460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.558535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.560455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 870/1753] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:25 2022] DELETE /identity/v3/projects/e66700e80d124f92a31eb05c97fa9555 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:25.561752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.563522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.627087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2d4af-8a02-471e-9705-1a8ded14fbad admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:25.631672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:25.632307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:25.632589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:25.632893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:25.633515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:25.635525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:25.784704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 871/1754] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:25 2022] POST /identity/v3/users => generated 309 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:25.785488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.787076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:25.799289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44f00d-ce28-48f2-bf1a-61e21b508782 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:25.803929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:25.804573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:25.804896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:25.805166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] PATH_INFO: `/v3/users/2ef4d822da75492facd22a9939926e9d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:25.806386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef4d822da75492facd22a9939926e9d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:25.808353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:25.836363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13c0b6305dc43678e4f9c3199085dde'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.837705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13c0b6305dc43678e4f9c3199085dde'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.954318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13c0b6305dc43678e4f9c3199085dde'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.954966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13c0b6305dc43678e4f9c3199085dde'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.963319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13c0b6305dc43678e4f9c3199085dde'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:25.980689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 884/1755] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:25 2022] DELETE /identity/v3/users/d13c0b6305dc43678e4f9c3199085dde => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:25.983704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:25.986368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:26.058542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde0c1d-8c4a-4a78-be09-83d1d1dfaafc admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:26.063174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:26.063838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:26.064078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:26.064346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:26.064987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:26.066652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:26.077354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 885/1756] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:26.078348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:26.080525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:26.135476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef4d822da75492facd22a9939926e9d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.141254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6864f6b-1994-41ef-819f-3fd934f42c46 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:26.145496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:26.146115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:26.146397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:26.146670 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:26.150561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef4d822da75492facd22a9939926e9d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.153670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef4d822da75492facd22a9939926e9d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.155268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:26.168725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef4d822da75492facd22a9939926e9d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.169555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef4d822da75492facd22a9939926e9d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.224240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:26.225765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:26.236292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 886/1757] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:26.236724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:26.249351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432556e-d62e-454d-8adc-81902a7885b9 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:26.255676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:26.256381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:26.256663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:26.256939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] PATH_INFO: `/v3/projects/18e4125d4f384ca0b8f5d24648b42b30/users/4c972d12f9df42859a7f3402b7135684/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:26.279063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e4125d4f384ca0b8f5d24648b42b30, user_id=4c972d12f9df42859a7f3402b7135684, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:26.282485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:26.313075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 872/1758] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:25 2022] DELETE /identity/v3/users/2ef4d822da75492facd22a9939926e9d => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:26.390353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 887/1759] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:26 2022] PUT /identity/v3/projects/18e4125d4f384ca0b8f5d24648b42b30/users/4c972d12f9df42859a7f3402b7135684/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:26.391634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:26.393230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:26.414038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:26.495141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74fb85-e319-4184-ad95-9dac85e201e8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:26.552620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:26.553249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:26.553528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:26.553801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:26.554451 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:26.556662 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:26.567272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 873/1760] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:26.571414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:26.582155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3607c70-eb03-4549-814f-d7b5d5ad7829 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:26.586914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:26.587759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:26.588069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:26.588341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] PATH_INFO: `/v3/projects/18e4125d4f384ca0b8f5d24648b42b30/users/4c972d12f9df42859a7f3402b7135684/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:26.590233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e4125d4f384ca0b8f5d24648b42b30, user_id=4c972d12f9df42859a7f3402b7135684, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:26.593005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:26.608285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:26.612569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 888/1761] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:26 2022] PUT /identity/v3/projects/18e4125d4f384ca0b8f5d24648b42b30/users/4c972d12f9df42859a7f3402b7135684/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:26.613778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:26.632045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3b36c-e489-4187-b35c-2e34ebbe590c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:26.715931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:26.716674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:26.716966 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:26.717817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] PATH_INFO: `/v3/projects/9bd260fc68d343b5a65a8d5f91960567` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:26.718427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd260fc68d343b5a65a8d5f91960567)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:26.720901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:26.894719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd260fc68d343b5a65a8d5f91960567', 'request_id': 'req-fe19d064-eafb-4cd3-a794-f52cadb5c445'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.901104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd260fc68d343b5a65a8d5f91960567', 'request_id': 'req-fe19d064-eafb-4cd3-a794-f52cadb5c445'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.908284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd260fc68d343b5a65a8d5f91960567', 'request_id': 'req-fe19d064-eafb-4cd3-a794-f52cadb5c445'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.913536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:26.916092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:26.928773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0630da-125c-42b0-a3d6-937e9d8f6f97 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:26.974541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:26.996590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] The token cache is being invalidate because project 9bd260fc68d343b5a65a8d5f91960567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:26.997062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:26.998375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe19d064-eafb-4cd3-a794-f52cadb5c445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:27.000666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 874/1762] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:26 2022] DELETE /identity/v3/projects/9bd260fc68d343b5a65a8d5f91960567 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:27.005492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6d1b-9e9c-4f5a-af15-4b0a08e40701 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.005780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6d1b-9e9c-4f5a-af15-4b0a08e40701 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.006058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6d1b-9e9c-4f5a-af15-4b0a08e40701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.016723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.017430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.017886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.018168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] PATH_INFO: `/v3/projects/d09c870eef314de1b233c385046faf3e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.019353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09c870eef314de1b233c385046faf3e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.021369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.036228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a61a6d1b-9e9c-4f5a-af15-4b0a08e40701 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:27.036228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a61a6d1b-9e9c-4f5a-af15-4b0a08e40701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.047604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 875/1763] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:27.057499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb89fc-2460-4eb7-af3b-856d5da4488f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.057787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb89fc-2460-4eb7-af3b-856d5da4488f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.063057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb89fc-2460-4eb7-af3b-856d5da4488f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.077961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-48bb89fc-2460-4eb7-af3b-856d5da4488f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:27.079168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-48bb89fc-2460-4eb7-af3b-856d5da4488f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.091875 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 876/1764] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:27.096331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb89fc-2460-4eb7-af3b-856d5da4488f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.106331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb89fc-2460-4eb7-af3b-856d5da4488f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.113116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.119439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.119439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.119439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.119439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:27.119439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.119439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.164787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 877/1765] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:11:27.166848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.182158 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547b2b-134f-4036-9624-69b6f76324bd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.195013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09c870eef314de1b233c385046faf3e', 'request_id': 'req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:27.208361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09c870eef314de1b233c385046faf3e', 'request_id': 'req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:27.232724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09c870eef314de1b233c385046faf3e', 'request_id': 'req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:27.284546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.285266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.285552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.286184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.286857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.288909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.324240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] The token cache is being invalidate because project d09c870eef314de1b233c385046faf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:27.324688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:27.325349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6d0e4b2f-41ab-4ebb-ae8d-be185c1149cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:27.326867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 889/1766] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:26 2022] DELETE /identity/v3/projects/d09c870eef314de1b233c385046faf3e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:27.331686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b88e-ec45-4755-bcd1-2b7719b31752 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.332121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b88e-ec45-4755-bcd1-2b7719b31752 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.332471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b88e-ec45-4755-bcd1-2b7719b31752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.387061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9928b88e-ec45-4755-bcd1-2b7719b31752 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:27.390318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9928b88e-ec45-4755-bcd1-2b7719b31752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.422795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 890/1767] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:27.422795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c55f3-b814-4f9b-aa11-1fca9369c99e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.423712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 878/1768] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/projects => generated 371 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:27.424019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c55f3-b814-4f9b-aa11-1fca9369c99e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.424019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c55f3-b814-4f9b-aa11-1fca9369c99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.424189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.441157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.470138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a45c55f3-b814-4f9b-aa11-1fca9369c99e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:27.472451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a45c55f3-b814-4f9b-aa11-1fca9369c99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.550317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f08f25-0df2-4ebb-98de-8c152336d784 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.556063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.556996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.557347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.557726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.558433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.560984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.569387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 891/1769] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:27.573816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c55f3-b814-4f9b-aa11-1fca9369c99e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.588692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c55f3-b814-4f9b-aa11-1fca9369c99e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.594973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.595830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.595944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.596166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.596810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:27.597111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.599260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.612759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 892/1770] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:11:27.616847 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.627445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c16fc6-0cb2-49c0-b414-ea1111b711a8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.632297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.632863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.633217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.633570 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.634262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.636161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.646631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 879/1771] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:27.651627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.661473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49cdae-4c9b-4fad-a3d5-f94317b9c899 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.666685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.667255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.667535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.667852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.668491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.670584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.684458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 880/1772] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:27.686157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.696132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217a0c8-b96c-4612-bd98-2ed04a815652 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.701653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.701808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.702130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.702413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.703050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.705416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.726133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 881/1773] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:27.728570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.740873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff102d-e9fd-429b-b07e-0d0109fd56e0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.746089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dda0ac-8b15-4f48-9052-89fe42da09b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.746835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dda0ac-8b15-4f48-9052-89fe42da09b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.747992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dda0ac-8b15-4f48-9052-89fe42da09b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.747992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dda0ac-8b15-4f48-9052-89fe42da09b5 admin admin] PATH_INFO: `/v3/projects/d07b62a38d324811a951f04028cf2b77/users/2be0803d7aa848ca991e56da69eb2e7a/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.765954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dda0ac-8b15-4f48-9052-89fe42da09b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07b62a38d324811a951f04028cf2b77, user_id=2be0803d7aa848ca991e56da69eb2e7a, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.768933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dda0ac-8b15-4f48-9052-89fe42da09b5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.778241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 893/1774] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:27.782424 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.849502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32673e-ac5f-4eea-9f35-7e92365fcfef admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.857169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.858309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.866544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.866544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.866544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:27.866544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:27.900668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 882/1775] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:27 2022] PUT /identity/v3/projects/d07b62a38d324811a951f04028cf2b77/users/2be0803d7aa848ca991e56da69eb2e7a/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:27.903157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.903443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.903718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.922841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:27.924085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.981840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 883/1776] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:27.993665 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.995337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:28.001018 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 894/1777] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:28.002856 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:28.021177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817a01-aac3-4b16-abfa-6096a49cc941 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:28.028717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:28.036026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.036026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.036026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.036026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:28.036026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:28.047729 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 895/1778] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:28.051432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:28.062315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c12e0-bb66-4328-bd14-8f59c5ed9e9e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:28.072488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:28.076494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.076863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.077219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfeea0-d298-418c-a0fc-4fc59c47358d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:28.077894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.079233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:28.081096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f55bff-ebe8-4d2c-a65e-4042ae8c8eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:28.081226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:28.081801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55bff-ebe8-4d2c-a65e-4042ae8c8eaa service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.082251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55bff-ebe8-4d2c-a65e-4042ae8c8eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.082595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55bff-ebe8-4d2c-a65e-4042ae8c8eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.094694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f55bff-ebe8-4d2c-a65e-4042ae8c8eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:28.096265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f55bff-ebe8-4d2c-a65e-4042ae8c8eaa service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:28.102116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 896/1779] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:28.102116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:28.107007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 884/1780] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:28.111397 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd463c3f-5b50-42ec-bbdf-5397a44a1ff0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:28.116249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:28.116953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.117228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.117498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] PATH_INFO: `/v3/projects/decf5d1a75104540a5955b930b990275/users/a60b6110604e454e97f1382035b91e5b/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.133421 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=decf5d1a75104540a5955b930b990275, user_id=a60b6110604e454e97f1382035b91e5b, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:28.136968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:28.299564 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 897/1781] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:28 2022] PUT /identity/v3/projects/decf5d1a75104540a5955b930b990275/users/a60b6110604e454e97f1382035b91e5b/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:28.304835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820fca-73a6-44f5-b6e1-c4a72bc9609c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.305329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820fca-73a6-44f5-b6e1-c4a72bc9609c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.305419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820fca-73a6-44f5-b6e1-c4a72bc9609c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.338180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-39820fca-73a6-44f5-b6e1-c4a72bc9609c None None] MFA Rules not processed for user `a60b6110604e454e97f1382035b91e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:28.339511 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-39820fca-73a6-44f5-b6e1-c4a72bc9609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:28.444090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 885/1782] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:28.449008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:28.535034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b1c43-ee98-4074-be69-2dfc939738c6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:28.537349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5df6-363f-4f09-a97f-ce7772ec99f9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.538381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5df6-363f-4f09-a97f-ce7772ec99f9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.538657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5df6-363f-4f09-a97f-ce7772ec99f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.544516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:28.545161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.545421 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.545707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.547201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:28.549265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:28.581866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-556b5df6-363f-4f09-a97f-ce7772ec99f9 None None] MFA Rules not processed for user `4c972d12f9df42859a7f3402b7135684`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:28.585735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-556b5df6-363f-4f09-a97f-ce7772ec99f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:28.708338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 886/1783] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:28.718833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 898/1784] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:28 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:28.722639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b5df6-363f-4f09-a97f-ce7772ec99f9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:28.748036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b5df6-363f-4f09-a97f-ce7772ec99f9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:28.872115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:28.872575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:28.872916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:28.873110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:28.875691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:28.878183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:29.041201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:29.043170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:29.060867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 887/1785] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:28 2022] POST /identity/v3/users => generated 299 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:29.064717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:29.075864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94732c-b64a-4be9-8dc7-0136e2c7908e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:29.080903 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:29.081498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:29.081810 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:29.083192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:29.083192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:29.084322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:29.095195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 888/1786] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:29.099184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:29.109749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89f151-239f-42ae-89c4-96130077282f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:29.113842 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:29.114483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:29.114848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:29.115194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:29.116180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac56204-3509-477a-a8be-a76464d6ebca admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:29.117127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:29.121644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:29.122304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:29.122582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:29.122854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] PATH_INFO: `/v3/projects/eba7a47186d14007be32666105436717/users/6a2abb95bb3041f49a52dd21ff770c91/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:29.149271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba7a47186d14007be32666105436717, user_id=6a2abb95bb3041f49a52dd21ff770c91, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:29.152242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:29.194541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:29.197461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:29.208307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 899/1787] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:29.290037 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 889/1788] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:29 2022] PUT /identity/v3/projects/eba7a47186d14007be32666105436717/users/6a2abb95bb3041f49a52dd21ff770c91/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:29.294655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:29.310162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf58b1-7955-4e2d-af44-26aa12c10832 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:29.372635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b8dfe-68e7-435a-a9b8-fcf000eef766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:29.373255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8dfe-68e7-435a-a9b8-fcf000eef766 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:29.373533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8dfe-68e7-435a-a9b8-fcf000eef766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:29.373805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8dfe-68e7-435a-a9b8-fcf000eef766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:29.374446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b8dfe-68e7-435a-a9b8-fcf000eef766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:29.376062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b8dfe-68e7-435a-a9b8-fcf000eef766 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:29.387825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 900/1789] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:29.392640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:29.403495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f0ce4-9e17-4fc9-a5ee-cc9caa281208 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:29.408404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9660b92-f72f-49f0-8120-55c932099cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:29.409153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9660b92-f72f-49f0-8120-55c932099cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:29.409501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9660b92-f72f-49f0-8120-55c932099cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:29.409866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9660b92-f72f-49f0-8120-55c932099cf8 admin admin] PATH_INFO: `/v3/projects/eba7a47186d14007be32666105436717/users/6a2abb95bb3041f49a52dd21ff770c91/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:29.412131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9660b92-f72f-49f0-8120-55c932099cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba7a47186d14007be32666105436717, user_id=6a2abb95bb3041f49a52dd21ff770c91, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:29.415069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9660b92-f72f-49f0-8120-55c932099cf8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:29.425648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 890/1790] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:29 2022] PUT /identity/v3/projects/eba7a47186d14007be32666105436717/users/6a2abb95bb3041f49a52dd21ff770c91/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:29.437102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:29.437393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:29.437667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:29.473948 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] MFA Rules not processed for user `6a2abb95bb3041f49a52dd21ff770c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:29.475184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:29.572698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 901/1791] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:29.720873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:30.072663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:30.073484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:30.073767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:30.127312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] MFA Rules not processed for user `2be0803d7aa848ca991e56da69eb2e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:30.128583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:30.193825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 891/1792] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:30.367493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:30.369557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:30.382957 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:30.426232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff54cc-abb7-4431-9b70-6fe34a0411a3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:30.430192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:30.430806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:30.431079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:30.431402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:30.478620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:30.481213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:30.494447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:30.496043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:30.505873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 902/1793] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:30.552817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee06f2-626f-4ba7-8fe2-0eee10d22c77 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:30.558834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:30.559190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:30.559471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:30.559748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] PATH_INFO: `/v3/users/64b0b9c929f8452380ef923190caefad` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:30.561015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b0b9c929f8452380ef923190caefad)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:30.563016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:30.627734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:30.637949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85098d-3a98-4db1-ac6d-40e4902b687d service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:30.641938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:30.642547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:30.642826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:30.643096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:30.651950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:30.653363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:30.662214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 903/1794] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:30.987599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b0b9c929f8452380ef923190caefad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:30.999934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b0b9c929f8452380ef923190caefad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.000610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b0b9c929f8452380ef923190caefad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.007547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b0b9c929f8452380ef923190caefad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.008187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b0b9c929f8452380ef923190caefad'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.166975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 892/1795] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:30 2022] DELETE /identity/v3/users/64b0b9c929f8452380ef923190caefad => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:31.416635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:31.429384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febffa-0f39-4492-a75b-d2962a9ea874 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:31.492518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:31.493373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:31.493801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:31.494145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] PATH_INFO: `/v3/projects/d66aec8ea88b4ac5ac107037bc85593d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:31.495413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d66aec8ea88b4ac5ac107037bc85593d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:31.497488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:31.627093 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd66aec8ea88b4ac5ac107037bc85593d', 'request_id': 'req-9198e549-ab44-4470-8f1e-25ea1dc18b08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.632802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd66aec8ea88b4ac5ac107037bc85593d', 'request_id': 'req-9198e549-ab44-4470-8f1e-25ea1dc18b08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.639681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd66aec8ea88b4ac5ac107037bc85593d', 'request_id': 'req-9198e549-ab44-4470-8f1e-25ea1dc18b08'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.750153 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] The token cache is being invalidate because project d66aec8ea88b4ac5ac107037bc85593d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:31.750623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.751602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9198e549-ab44-4470-8f1e-25ea1dc18b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:31.753502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 904/1796] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:31 2022] DELETE /identity/v3/projects/d66aec8ea88b4ac5ac107037bc85593d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:31.757397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:31.759623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:31.773369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7adb7-a599-49c5-8238-72e56aeed258 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:31.841706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:31.842266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:31.842545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:31.842818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] PATH_INFO: `/v3/users/4c972d12f9df42859a7f3402b7135684` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:31.843958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c972d12f9df42859a7f3402b7135684)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:31.845883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:32.116236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c972d12f9df42859a7f3402b7135684'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.128890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c972d12f9df42859a7f3402b7135684'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.134849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c972d12f9df42859a7f3402b7135684'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.137156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c972d12f9df42859a7f3402b7135684'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.137785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c972d12f9df42859a7f3402b7135684'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.227955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 893/1797] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:31 2022] DELETE /identity/v3/users/4c972d12f9df42859a7f3402b7135684 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:32.422765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c7861-47d2-4175-8e11-93b2f55195c8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:32.423300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c7861-47d2-4175-8e11-93b2f55195c8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:32.423430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c7861-47d2-4175-8e11-93b2f55195c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:32.443411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2a0c7861-47d2-4175-8e11-93b2f55195c8 None None] MFA Rules not processed for user `2be0803d7aa848ca991e56da69eb2e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:32.445316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2a0c7861-47d2-4175-8e11-93b2f55195c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:32.518527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 905/1798] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:32.530119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:32.531586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:32.629387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c372f-8fcb-4970-a885-3ca154f4bdc7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:32.638862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:32.638862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:32.638862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:32.638862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] PATH_INFO: `/v3/projects/18e4125d4f384ca0b8f5d24648b42b30` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:32.639703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e4125d4f384ca0b8f5d24648b42b30)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:32.642299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:32.785574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e4125d4f384ca0b8f5d24648b42b30', 'request_id': 'req-429c901e-effd-4e5b-93b3-8bb7215444ea'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.797879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e4125d4f384ca0b8f5d24648b42b30', 'request_id': 'req-429c901e-effd-4e5b-93b3-8bb7215444ea'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.808433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e4125d4f384ca0b8f5d24648b42b30', 'request_id': 'req-429c901e-effd-4e5b-93b3-8bb7215444ea'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.826671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f400a-84ef-4336-8fe2-355e64eda969 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:32.826964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f400a-84ef-4336-8fe2-355e64eda969 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:32.827239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f400a-84ef-4336-8fe2-355e64eda969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1a6f400a-84ef-4336-8fe2-355e64eda969 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:11:32.845300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:32.850653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:11:32.853310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 906/1799] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:11:32 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 21 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Oct 26 11:11:32.853310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f400a-84ef-4336-8fe2-355e64eda969 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:32.853310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1a6f400a-84ef-4336-8fe2-355e64eda969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:32.908131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:32.909664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] The token cache is being invalidate because project 18e4125d4f384ca0b8f5d24648b42b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:32.910081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.910731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-429c901e-effd-4e5b-93b3-8bb7215444ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:32.916010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 894/1800] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:32 2022] DELETE /identity/v3/projects/18e4125d4f384ca0b8f5d24648b42b30 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:32.916010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c397d-9d51-4ea8-b1ec-986063153e4f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:32.916010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c397d-9d51-4ea8-b1ec-986063153e4f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:32.916010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c397d-9d51-4ea8-b1ec-986063153e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:32.935704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f08c397d-9d51-4ea8-b1ec-986063153e4f None None] MFA Rules not processed for user `2be0803d7aa848ca991e56da69eb2e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:32.936263 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f400a-84ef-4336-8fe2-355e64eda969 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:32.936934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f08c397d-9d51-4ea8-b1ec-986063153e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:32.940000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08309784-72c1-4c69-8849-b0045374bf80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:32.941355 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309784-72c1-4c69-8849-b0045374bf80 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:32.941636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309784-72c1-4c69-8849-b0045374bf80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:32.941908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08309784-72c1-4c69-8849-b0045374bf80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:32.944470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-08309784-72c1-4c69-8849-b0045374bf80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:32.998058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 895/1801] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:32.999308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5e4a4-3571-4901-8e16-1735c6f5698a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:32.999594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5e4a4-3571-4901-8e16-1735c6f5698a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:32.999895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5e4a4-3571-4901-8e16-1735c6f5698a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.019772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08309784-72c1-4c69-8849-b0045374bf80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.021240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08309784-72c1-4c69-8849-b0045374bf80 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:33.023078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5aa5e4a4-3571-4901-8e16-1735c6f5698a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:33.025407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5aa5e4a4-3571-4901-8e16-1735c6f5698a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:33.039213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 907/1802] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:33.096286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 896/1803] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:33.106683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862896f7-98ac-4310-b17c-685e91e58837 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.107294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862896f7-98ac-4310-b17c-685e91e58837 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.107966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862896f7-98ac-4310-b17c-685e91e58837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.129566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-862896f7-98ac-4310-b17c-685e91e58837 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:33.130875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-862896f7-98ac-4310-b17c-685e91e58837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:33.139626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 908/1804] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:33.143970 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5e4a4-3571-4901-8e16-1735c6f5698a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:33.154496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5e4a4-3571-4901-8e16-1735c6f5698a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:33.159722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ed75-19d8-4163-a482-082d55f63146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:33.160479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ed75-19d8-4163-a482-082d55f63146 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.160847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ed75-19d8-4163-a482-082d55f63146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.161186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ed75-19d8-4163-a482-082d55f63146 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.161900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ed75-19d8-4163-a482-082d55f63146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:33.162271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ed75-19d8-4163-a482-082d55f63146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.163989 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ed75-19d8-4163-a482-082d55f63146 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:33.176615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 897/1805] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:11:33.180883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862896f7-98ac-4310-b17c-685e91e58837 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:33.190411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862896f7-98ac-4310-b17c-685e91e58837 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:33.196367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:33.197015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.197295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.197562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.198958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.208460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:33.241429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa4984-8133-4177-b9e2-8d6a2017d5e4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.243405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa4984-8133-4177-b9e2-8d6a2017d5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.243405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa4984-8133-4177-b9e2-8d6a2017d5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-89fa4984-8133-4177-b9e2-8d6a2017d5e4 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:11:33.248637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:11:33.249582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 11:11:33.250520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 898/1806] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:11:33 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 9 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:11:33.367326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 909/1807] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:33 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:33.369474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa4984-8133-4177-b9e2-8d6a2017d5e4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:33.434634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa4984-8133-4177-b9e2-8d6a2017d5e4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:33.439070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:33.439704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.440012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.440284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.440919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.442874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:33.563249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 899/1808] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:33 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:33.568404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:33.578740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2daeda2-70f8-4c89-bd83-b8f2d3d21289 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:33.583327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:33.583991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.584267 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.584535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.585168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.587414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:33.597473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 910/1809] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:33.601350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:33.611330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:33.613777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539c025-d06b-4eb6-884f-ace757e63f37 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:33.619072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646482b3-393d-4249-9ce0-f798e037a853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:33.620374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646482b3-393d-4249-9ce0-f798e037a853 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.620374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646482b3-393d-4249-9ce0-f798e037a853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.620374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646482b3-393d-4249-9ce0-f798e037a853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.620919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646482b3-393d-4249-9ce0-f798e037a853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.622521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646482b3-393d-4249-9ce0-f798e037a853 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:33.633066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 900/1810] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:33.637226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:33.653023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418e75c-bac2-407f-af3b-f49b80337a77 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:33.657800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:33.658470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.658750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.659046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] PATH_INFO: `/v3/projects/354f089c0bce47fc80e80ee648ecaf84/users/a205184e1571402f9e7e722eca113819/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.678558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354f089c0bce47fc80e80ee648ecaf84, user_id=a205184e1571402f9e7e722eca113819, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.681426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:33.779523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 911/1811] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:33 2022] PUT /identity/v3/projects/354f089c0bce47fc80e80ee648ecaf84/users/a205184e1571402f9e7e722eca113819/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:33.790080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f94f1-ed76-4997-aadc-c61551bfbee8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.790566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f94f1-ed76-4997-aadc-c61551bfbee8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.790762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f94f1-ed76-4997-aadc-c61551bfbee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.818662 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-119f94f1-ed76-4997-aadc-c61551bfbee8 None None] MFA Rules not processed for user `a205184e1571402f9e7e722eca113819`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:33.819915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-119f94f1-ed76-4997-aadc-c61551bfbee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:33.884947 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 901/1812] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:33.889525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:33.954106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37fc9c-6df6-4bfa-95fd-5120496f8605 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:33.959620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:33.960325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:33.960668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:33.961008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:33.961842 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:33.964484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:34.112073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 912/1813] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:33 2022] POST /identity/v3/projects => generated 363 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:11:34.115897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f94f1-ed76-4997-aadc-c61551bfbee8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:34.127503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f94f1-ed76-4997-aadc-c61551bfbee8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:34.204063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:34.204063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:34.204063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:34.204063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:34.204413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:34.204413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:34.301581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 902/1814] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:34 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:11:34.303280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:34.313646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b474-87fb-4a4c-87d1-b0843f49625e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:34.318365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:34.318965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:34.319269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:34.319545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:34.320258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:34.321870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:34.332264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 913/1815] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:34.336314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:34.348669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d64d8-434b-4e1c-b941-4a1d6f209c1a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:34.354046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:34.354701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:34.354978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:34.355249 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] PATH_INFO: `/v3/projects/08b195fba35b487b83117cfde3f9ca2f/users/98f37a7f2d9f48699761ab6284dbd2ff/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:34.370242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b195fba35b487b83117cfde3f9ca2f, user_id=98f37a7f2d9f48699761ab6284dbd2ff, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:34.372952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:34.544817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 903/1816] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:34 2022] PUT /identity/v3/projects/08b195fba35b487b83117cfde3f9ca2f/users/98f37a7f2d9f48699761ab6284dbd2ff/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:34.548877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:34.559466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37835e-0429-4a1f-911e-dbfad66b279c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:34.614028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fe812-e3a9-4dc4-8c4d-360b2028ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:34.614660 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fe812-e3a9-4dc4-8c4d-360b2028ab3a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:34.614895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fe812-e3a9-4dc4-8c4d-360b2028ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:34.615170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fe812-e3a9-4dc4-8c4d-360b2028ab3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:34.615819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fe812-e3a9-4dc4-8c4d-360b2028ab3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:34.617423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fe812-e3a9-4dc4-8c4d-360b2028ab3a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:34.631056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 914/1817] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:34.632535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:34.643594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a94a3-7280-4625-a3e1-942ce58560af admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:34.649864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:34.650590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:34.650875 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:34.651149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] PATH_INFO: `/v3/projects/08b195fba35b487b83117cfde3f9ca2f/users/98f37a7f2d9f48699761ab6284dbd2ff/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:34.657649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b195fba35b487b83117cfde3f9ca2f, user_id=98f37a7f2d9f48699761ab6284dbd2ff, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:34.657649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:34.668995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 904/1818] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:34 2022] PUT /identity/v3/projects/08b195fba35b487b83117cfde3f9ca2f/users/98f37a7f2d9f48699761ab6284dbd2ff/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:34.679042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:34.679331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:34.679604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:34.710292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] MFA Rules not processed for user `98f37a7f2d9f48699761ab6284dbd2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:34.712054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:34.804344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 915/1819] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:34.883560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:34.885340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:34.946621 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3a247-2bd4-4204-b0b0-a908342f0c1f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:34.950992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ca5d9-756b-4dc4-be7e-c81222be4ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:34.951816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca5d9-756b-4dc4-be7e-c81222be4ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:34.952149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca5d9-756b-4dc4-be7e-c81222be4ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:34.952485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca5d9-756b-4dc4-be7e-c81222be4ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:34.959173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:34.960979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:34.967847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ca5d9-756b-4dc4-be7e-c81222be4ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:34.969723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ca5d9-756b-4dc4-be7e-c81222be4ee9 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:34.983058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 905/1820] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:35.028893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2860e9-90d9-472e-b921-a7fa6de8c29c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:35.033410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:35.034044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:35.034317 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:35.034587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] PATH_INFO: `/v3/users/21db9e77db0c491090e3b7d9188d8ada` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:35.035709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=21db9e77db0c491090e3b7d9188d8ada)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:35.037688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:35.243437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21db9e77db0c491090e3b7d9188d8ada'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:35.244100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21db9e77db0c491090e3b7d9188d8ada'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:35.378419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21db9e77db0c491090e3b7d9188d8ada'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:35.379051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21db9e77db0c491090e3b7d9188d8ada'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:35.393513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21db9e77db0c491090e3b7d9188d8ada'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:35.415895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 916/1821] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:34 2022] DELETE /identity/v3/users/21db9e77db0c491090e3b7d9188d8ada => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:35.420869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:35.421156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:35.421431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:35.436094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:35.440675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:35.441890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:35.500705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 906/1822] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:35.511595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:35.513057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:35.575455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4050-acdd-419a-ac5f-436279bdb9db admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:35.579148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:35.579757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:35.580150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:35.580425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:35.590368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:35.591828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:35.601376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 917/1823] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:35.779437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:36.043922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:36.045506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:36.058463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473165f-09f5-48d0-9edb-96847ba188fe None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:36.063964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:36.065074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:36.065333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:36.065683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] PATH_INFO: `/v3/projects/149e2dc638074494ba2d8b02d47f7382` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:36.067034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=149e2dc638074494ba2d8b02d47f7382)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:36.069473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:36.196440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149e2dc638074494ba2d8b02d47f7382', 'request_id': 'req-cd54eada-fb83-42f7-92d1-72ac0f23d21c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.201884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149e2dc638074494ba2d8b02d47f7382', 'request_id': 'req-cd54eada-fb83-42f7-92d1-72ac0f23d21c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.208616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149e2dc638074494ba2d8b02d47f7382', 'request_id': 'req-cd54eada-fb83-42f7-92d1-72ac0f23d21c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.321768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:36.323458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:36.332489 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] The token cache is being invalidate because project 149e2dc638074494ba2d8b02d47f7382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:36.332864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.333599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.337500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 907/1824] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:36 2022] DELETE /identity/v3/projects/149e2dc638074494ba2d8b02d47f7382 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:36.337624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d50df-a596-47e5-81c8-c8181e9d9ad2 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:36.345455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:36.347163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:36.424364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:36.425143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:36.425768 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:36.426450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] PATH_INFO: `/v3/users/a205184e1571402f9e7e722eca113819` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:36.430555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a205184e1571402f9e7e722eca113819)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:36.434486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:36.442371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54eada-fb83-42f7-92d1-72ac0f23d21c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:36.447555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:36.448375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:36.448724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:36.449228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] PATH_INFO: `/v3/users/b913ae43cd9e431fbc2dab77e91ce902` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:36.450603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b913ae43cd9e431fbc2dab77e91ce902)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:36.452631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:36.587650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:36.719820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a205184e1571402f9e7e722eca113819'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.720089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b913ae43cd9e431fbc2dab77e91ce902'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.720579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a205184e1571402f9e7e722eca113819'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.733901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b913ae43cd9e431fbc2dab77e91ce902'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.734688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b913ae43cd9e431fbc2dab77e91ce902'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.741871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b913ae43cd9e431fbc2dab77e91ce902'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.742998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b913ae43cd9e431fbc2dab77e91ce902'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.819974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a205184e1571402f9e7e722eca113819'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.821890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a205184e1571402f9e7e722eca113819'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.840649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42c88514-1700-4851-b510-f7d3781bc99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a205184e1571402f9e7e722eca113819'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:36.876535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 918/1825] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:36 2022] DELETE /identity/v3/users/a205184e1571402f9e7e722eca113819 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:36.881123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:36.881419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:36.881690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:36.903421 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:36.904673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:36.963172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 919/1826] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:36.973194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:36.974662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:36.983742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 908/1827] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:36 2022] DELETE /identity/v3/users/b913ae43cd9e431fbc2dab77e91ce902 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:37.040255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32191848-f93b-4a5e-a8f5-ff1c8c8704c7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:37.090170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:37.090868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:37.091051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:37.091389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:37.155080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:37.156581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:37.167487 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 920/1828] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:37.499836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:37.501982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:37.518883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6f833-d731-4d93-910b-0fedffd3f3b8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:37.525261 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:37.526297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:37.527139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:37.527430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] PATH_INFO: `/v3/projects/3e49d64964d643bf87fddbdaaf5cf0e1` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:37.528964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e49d64964d643bf87fddbdaaf5cf0e1)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:37.531039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:37.642046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e49d64964d643bf87fddbdaaf5cf0e1', 'request_id': 'req-32320623-d965-45e6-b9b1-b7deb9b2805d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.647460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e49d64964d643bf87fddbdaaf5cf0e1', 'request_id': 'req-32320623-d965-45e6-b9b1-b7deb9b2805d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.649604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:37.651228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:37.655613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e49d64964d643bf87fddbdaaf5cf0e1', 'request_id': 'req-32320623-d965-45e6-b9b1-b7deb9b2805d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.664929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6ee78-715a-45e9-b723-15ef636ecff0 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:37.722839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:37.723468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:37.723743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:37.724049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] PATH_INFO: `/v3/projects/354f089c0bce47fc80e80ee648ecaf84` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:37.725189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=354f089c0bce47fc80e80ee648ecaf84)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:37.727683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:37.766347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] The token cache is being invalidate because project 3e49d64964d643bf87fddbdaaf5cf0e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:37.766637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.767328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-32320623-d965-45e6-b9b1-b7deb9b2805d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.768843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 909/1829] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:37 2022] DELETE /identity/v3/projects/3e49d64964d643bf87fddbdaaf5cf0e1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:37.773508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08639a56-cc01-4851-8c03-fa529c063343 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:37.773949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08639a56-cc01-4851-8c03-fa529c063343 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:37.774797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08639a56-cc01-4851-8c03-fa529c063343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:37.795242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-08639a56-cc01-4851-8c03-fa529c063343 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:37.796496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-08639a56-cc01-4851-8c03-fa529c063343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:37.804421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 910/1830] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:37.816510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6be561-e9ec-494b-8dfd-3a42b8719779 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:37.816791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6be561-e9ec-494b-8dfd-3a42b8719779 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:37.817060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6be561-e9ec-494b-8dfd-3a42b8719779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:37.836737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ae6be561-e9ec-494b-8dfd-3a42b8719779 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:37.838015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ae6be561-e9ec-494b-8dfd-3a42b8719779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:37.845725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 911/1831] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:37.849660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6be561-e9ec-494b-8dfd-3a42b8719779 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:37.859023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6be561-e9ec-494b-8dfd-3a42b8719779 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:37.870850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354f089c0bce47fc80e80ee648ecaf84', 'request_id': 'req-3c6fe732-7f33-45bb-af00-1f57a238fbaf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.879827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354f089c0bce47fc80e80ee648ecaf84', 'request_id': 'req-3c6fe732-7f33-45bb-af00-1f57a238fbaf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.888446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354f089c0bce47fc80e80ee648ecaf84', 'request_id': 'req-3c6fe732-7f33-45bb-af00-1f57a238fbaf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:37.920334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:37.920948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:37.921228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:37.921558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:37.923075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:37.923075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:37.924194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:37.938214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 912/1832] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:11:37.942097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:37.952245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c0ff-5d67-4a07-937b-26c53e4bf7a4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:37.957312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:37.957906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:37.958185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:37.958456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:37.959084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:37.961074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:38.006926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] The token cache is being invalidate because project 354f089c0bce47fc80e80ee648ecaf84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:38.007150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:38.007827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:38.009910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 921/1833] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:37 2022] DELETE /identity/v3/projects/354f089c0bce47fc80e80ee648ecaf84 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:38.013804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:38.015089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:38.018809 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:38.027777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe732-7f33-45bb-af00-1f57a238fbaf admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:38.032459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:38.033172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:38.033430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:38.033706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] PATH_INFO: `/v3/users/98f37a7f2d9f48699761ab6284dbd2ff` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:38.034887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f37a7f2d9f48699761ab6284dbd2ff)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:38.036844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:38.077629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 913/1834] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:37 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:11:38.081781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:38.083358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:38.145395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667d42-3705-4df7-81b5-6715c897dc3c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:38.150214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:38.150817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:38.151095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:38.151364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:38.152019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:38.153933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:38.368145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f37a7f2d9f48699761ab6284dbd2ff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:38.369012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f37a7f2d9f48699761ab6284dbd2ff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:38.479882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 914/1835] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:38 2022] POST /identity/v3/users => generated 289 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:11:38.480939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:38.490811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634f2e6-5117-4ece-a1e3-7902cf305dc3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:38.496553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:38.497169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:38.497528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:38.497859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:38.498528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:38.500242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:38.509585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f37a7f2d9f48699761ab6284dbd2ff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:38.511129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 915/1836] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:38.511438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:38.514657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f37a7f2d9f48699761ab6284dbd2ff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:38.515310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:38.517029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:38.523212 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f37a7f2d9f48699761ab6284dbd2ff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:38.535381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321580f-1c90-4ea2-bacc-30e2977cb9d1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:38.540987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d30ec-4552-4951-a4bc-9997677c2ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:38.541598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d30ec-4552-4951-a4bc-9997677c2ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:38.541879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d30ec-4552-4951-a4bc-9997677c2ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:38.542150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d30ec-4552-4951-a4bc-9997677c2ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:38.542793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d30ec-4552-4951-a4bc-9997677c2ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:38.544440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d30ec-4552-4951-a4bc-9997677c2ddc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:38.551031 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 922/1837] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:38 2022] DELETE /identity/v3/users/98f37a7f2d9f48699761ab6284dbd2ff => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:38.565344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 916/1838] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:38.574440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:38.640710 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db867468-da2e-4974-aee9-5a29e4ae507c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:38.645324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:38.645990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:38.646266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:38.646536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] PATH_INFO: `/v3/projects/c63543222fe8446dbad0a95ac6ea313a/users/3f472eebdc9047bcbb32eb511cf5dd80/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:38.662808 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63543222fe8446dbad0a95ac6ea313a, user_id=3f472eebdc9047bcbb32eb511cf5dd80, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:38.668138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:38.801759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 923/1839] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:38 2022] PUT /identity/v3/projects/c63543222fe8446dbad0a95ac6ea313a/users/3f472eebdc9047bcbb32eb511cf5dd80/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:38.812551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:38.813167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:38.813445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:38.859602 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] MFA Rules not processed for user `3f472eebdc9047bcbb32eb511cf5dd80`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:38.861478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:38.934550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 917/1840] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:38.946402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.023099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5110fb4-e051-4038-9aa1-d0c5d15bc02d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.028668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.029407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.029693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.029967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.030616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.032597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.129178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.130739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:39.150570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc274-8f6b-40be-9ea5-b274e23d5476 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.162019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.162704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.162979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.163248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] PATH_INFO: `/v3/projects/08b195fba35b487b83117cfde3f9ca2f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.167717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 924/1841] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:38 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:39.167848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b195fba35b487b83117cfde3f9ca2f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.167848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.170974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.236954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65f385-4734-4188-b256-35daef797f0d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.241627 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.242214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.243119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.243391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.244049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.246036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.369791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b195fba35b487b83117cfde3f9ca2f', 'request_id': 'req-1e641e00-2fd1-4f1e-acec-06e0fa312743'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:39.378756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b195fba35b487b83117cfde3f9ca2f', 'request_id': 'req-1e641e00-2fd1-4f1e-acec-06e0fa312743'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:39.386117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b195fba35b487b83117cfde3f9ca2f', 'request_id': 'req-1e641e00-2fd1-4f1e-acec-06e0fa312743'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:39.386923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 925/1842] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:39.388561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.401622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98ddad-ba6e-424f-b97e-d51694929c0e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.459272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.459924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.460202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.460473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.461141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.462734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.473249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 926/1843] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:39.478136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.488064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6626e-17bb-40b4-977c-08591b2b6201 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.493836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.494608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.495048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.495527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.496350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.498336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.512169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 927/1844] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:39.516090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.526782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69255-2068-405a-b34b-148f5853d417 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.531687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb45f-cddd-43af-a7ab-f7731089d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.532400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb45f-cddd-43af-a7ab-f7731089d3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.532749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb45f-cddd-43af-a7ab-f7731089d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.533254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb45f-cddd-43af-a7ab-f7731089d3bf admin admin] PATH_INFO: `/v3/projects/7e39058b07d94acc86b50dc1fdd7c64f/users/cd1f4445774846debe33796a20402d65/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.549005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cb45f-cddd-43af-a7ab-f7731089d3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e39058b07d94acc86b50dc1fdd7c64f, user_id=cd1f4445774846debe33796a20402d65, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.551644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cb45f-cddd-43af-a7ab-f7731089d3bf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.566417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] The token cache is being invalidate because project 08b195fba35b487b83117cfde3f9ca2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:39.566876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:39.567483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1e641e00-2fd1-4f1e-acec-06e0fa312743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:39.569145 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 918/1845] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:39 2022] DELETE /identity/v3/projects/08b195fba35b487b83117cfde3f9ca2f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:39.574153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a134d-94e5-4fe0-8a6b-deea72f533d2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.574435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a134d-94e5-4fe0-8a6b-deea72f533d2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.574706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a134d-94e5-4fe0-8a6b-deea72f533d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.596041 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-320a134d-94e5-4fe0-8a6b-deea72f533d2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:39.597310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-320a134d-94e5-4fe0-8a6b-deea72f533d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:39.606597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 919/1846] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:39.615172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d0966-0173-4417-b55b-193ceb534e55 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.615455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d0966-0173-4417-b55b-193ceb534e55 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.615727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d0966-0173-4417-b55b-193ceb534e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.637188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-085d0966-0173-4417-b55b-193ceb534e55 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:39.638328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-085d0966-0173-4417-b55b-193ceb534e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:39.647139 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 920/1847] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:39.653071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 928/1848] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:39 2022] PUT /identity/v3/projects/7e39058b07d94acc86b50dc1fdd7c64f/users/cd1f4445774846debe33796a20402d65/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:39.654855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d0966-0173-4417-b55b-193ceb534e55 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.665720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d0966-0173-4417-b55b-193ceb534e55 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.670077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e9901-ea72-4ca4-98a8-38aaa6bb1a11 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.670707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e9901-ea72-4ca4-98a8-38aaa6bb1a11 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.670707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e9901-ea72-4ca4-98a8-38aaa6bb1a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.722070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a39e9901-ea72-4ca4-98a8-38aaa6bb1a11 None None] MFA Rules not processed for user `cd1f4445774846debe33796a20402d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:39.723291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a39e9901-ea72-4ca4-98a8-38aaa6bb1a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:39.764791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e0314-13be-4c09-9a43-0b0da535bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.765417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e0314-13be-4c09-9a43-0b0da535bd7c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.765700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e0314-13be-4c09-9a43-0b0da535bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.765977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e0314-13be-4c09-9a43-0b0da535bd7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.766662 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e0314-13be-4c09-9a43-0b0da535bd7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:39.766962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e0314-13be-4c09-9a43-0b0da535bd7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.768694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e0314-13be-4c09-9a43-0b0da535bd7c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.781429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 921/1849] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:39.782679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.782971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.783383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.799934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 929/1850] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:39.804279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e9901-ea72-4ca4-98a8-38aaa6bb1a11 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.816130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e9901-ea72-4ca4-98a8-38aaa6bb1a11 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.822111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.822727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.823007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.823277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.823965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.826309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.857329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:39.858626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:39.924640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 922/1851] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:39.925628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:39.927076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:39.943314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da510bc4-922d-4deb-aa62-52ec9b007540 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:39.948186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:39.948792 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:39.949071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:39.949374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:39.950127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:39.952063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:39.980930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 930/1852] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:39.985984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.063497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac561e-2c1e-4021-bb05-e20d84c86b73 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.068543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.069551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.069934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.070305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.072324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.074351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.181990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 923/1853] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/projects => generated 377 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:40.183242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.209460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 931/1854] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:39 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:40.214387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.224616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cadc02-d840-4d58-a458-eb7883f4b533 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.247813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039e5d-0d91-43fc-a7b7-bd40ff022cce admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.253219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.253911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.254190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.254457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.255115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.257107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.282887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.283543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.283822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.284165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.284806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.287074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.298581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 932/1855] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:40.299469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.301150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:40.378762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 924/1856] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:40 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:11:40.380023 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.395226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1080f-fc74-43e0-809e-7de3824d6817 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.396216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71fc9-6453-4f56-b723-bb0eeba39de9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.396569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.397228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.397487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.397766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.398443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.400800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.403622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.404376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.404810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.405176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.411666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 925/1857] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:40.416450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.429429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1c06d-6bb0-44ef-a4cc-b59dd52ba337 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.440905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.444202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.444563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.445181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.446494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.448284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.464457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 926/1858] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:40.464457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.474214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dbf7f-a51c-435b-8fb8-cdfbc21889af admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.480367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badd676-7764-447a-b40c-e6a79ee96633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.481155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd676-7764-447a-b40c-e6a79ee96633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.481435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd676-7764-447a-b40c-e6a79ee96633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.481893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd676-7764-447a-b40c-e6a79ee96633 admin admin] PATH_INFO: `/v3/projects/1fc325a95bfd4a71b4b1e4fc107caa09/users/e770e3cdf5de4d25bdfc614684c408ae/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.489291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.491564 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.498745 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badd676-7764-447a-b40c-e6a79ee96633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc325a95bfd4a71b4b1e4fc107caa09, user_id=e770e3cdf5de4d25bdfc614684c408ae, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.500793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 933/1859] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:11:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:11:40.508197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badd676-7764-447a-b40c-e6a79ee96633 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.513722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.551835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c3510-25d6-47e6-a0eb-32f71685fcbf service swift] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.559216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.559994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.560246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.560458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] PATH_INFO: `/v3/projects/268e09dd0c264c56b37b7248753de41e/users/bcf73b0932a743e3b764bf52e01e4e21/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.589605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268e09dd0c264c56b37b7248753de41e, user_id=bcf73b0932a743e3b764bf52e01e4e21, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.592709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.644052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 927/1860] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:40 2022] PUT /identity/v3/projects/1fc325a95bfd4a71b4b1e4fc107caa09/users/e770e3cdf5de4d25bdfc614684c408ae/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:40.657265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b39b-f178-4368-8d37-a0673f23632c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.657547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b39b-f178-4368-8d37-a0673f23632c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.657817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b39b-f178-4368-8d37-a0673f23632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.714483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-00f7b39b-f178-4368-8d37-a0673f23632c None None] MFA Rules not processed for user `e770e3cdf5de4d25bdfc614684c408ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:40.716642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-00f7b39b-f178-4368-8d37-a0673f23632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:40.722320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 934/1861] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:40 2022] PUT /identity/v3/projects/268e09dd0c264c56b37b7248753de41e/users/bcf73b0932a743e3b764bf52e01e4e21/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:40.726206 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.828264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce4766-d6ef-449e-bc29-7ca0c7be2d80 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.874172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.874172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.874172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.874172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.874586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.874586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.879227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 928/1862] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:40.881146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 935/1863] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:40.885485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.887657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7b39b-f178-4368-8d37-a0673f23632c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:40.896507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cc5a0-3da7-4126-aeee-eb63feef2982 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.905479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c5490-b745-464b-8d5b-5043ca3d5a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.905830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7b39b-f178-4368-8d37-a0673f23632c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:40.906154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c5490-b745-464b-8d5b-5043ca3d5a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.906418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c5490-b745-464b-8d5b-5043ca3d5a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.906692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c5490-b745-464b-8d5b-5043ca3d5a22 admin admin] PATH_INFO: `/v3/projects/268e09dd0c264c56b37b7248753de41e/users/bcf73b0932a743e3b764bf52e01e4e21/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.910418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c5490-b745-464b-8d5b-5043ca3d5a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268e09dd0c264c56b37b7248753de41e, user_id=bcf73b0932a743e3b764bf52e01e4e21, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.913766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c5490-b745-464b-8d5b-5043ca3d5a22 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.916666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:40.925502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.925502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.925502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:40.925502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:40.925502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:40.959937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 936/1864] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:40 2022] PUT /identity/v3/projects/268e09dd0c264c56b37b7248753de41e/users/bcf73b0932a743e3b764bf52e01e4e21/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:40.975002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da024-ecb7-4dba-ade3-cd7408b0614d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:40.977776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da024-ecb7-4dba-ade3-cd7408b0614d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:40.982903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da024-ecb7-4dba-ade3-cd7408b0614d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.023839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-af4da024-ecb7-4dba-ade3-cd7408b0614d None None] MFA Rules not processed for user `bcf73b0932a743e3b764bf52e01e4e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:41.025683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-af4da024-ecb7-4dba-ade3-cd7408b0614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:41.136180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 937/1865] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:41.151199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 929/1866] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:40 2022] POST /identity/v3/projects => generated 363 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:41.164257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4da024-ecb7-4dba-ade3-cd7408b0614d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:41.252439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4da024-ecb7-4dba-ade3-cd7408b0614d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:41.328093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:41.329825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:41.348221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:41.349144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.349633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.350117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.350977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:41.354227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:41.434161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ea052-a5e8-4536-806a-99c697ae90da admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:41.439535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:41.440548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.440876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.441105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.483590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 938/1867] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:41 2022] POST /identity/v3/users => generated 288 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:41.487569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:41.513663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ac6d-770e-400f-a3fe-f2fc713caf8b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:41.521012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:41.523180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:41.525843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:41.527649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.527649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.527649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.528154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:41.541529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:41.542625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 930/1868] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:41.562138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 939/1869] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:41.563463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:41.578350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5079c-4e6c-43d6-9881-8c2df6250cd9 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:41.580949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:41.585757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:41.586532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.586837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.587514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] PATH_INFO: `/v3/projects/625fa2d3f13a4350a9e2805f96c00d13/users/ce9e640586014d90af9c351ccc7c094d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.637471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625fa2d3f13a4350a9e2805f96c00d13, user_id=ce9e640586014d90af9c351ccc7c094d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:41.638982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:41.748893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:41.764595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843c439-296c-4af7-85c8-0cafe07463df admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:41.768571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:41.769190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.769467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.769798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.771554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:41.813908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 931/1870] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:41 2022] PUT /identity/v3/projects/625fa2d3f13a4350a9e2805f96c00d13/users/ce9e640586014d90af9c351ccc7c094d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:41.818022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:41.866297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:41.868059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:41.891667 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 940/1871] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:41.950782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b6cc5-201f-4059-a329-4b82b76bfb33 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:41.969255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405d8f2-fc39-4d41-b207-2b0fbbeafb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:41.970185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d8f2-fc39-4d41-b207-2b0fbbeafb14 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.970585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d8f2-fc39-4d41-b207-2b0fbbeafb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.970924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405d8f2-fc39-4d41-b207-2b0fbbeafb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.971668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405d8f2-fc39-4d41-b207-2b0fbbeafb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:41.974053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405d8f2-fc39-4d41-b207-2b0fbbeafb14 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:41.991902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 932/1872] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:41.996544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:42.009406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c8b09-9364-4792-885a-3d6dff49b58b service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:42.017619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:42.018930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:42.019211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:42.019532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] PATH_INFO: `/v3/projects/625fa2d3f13a4350a9e2805f96c00d13/users/ce9e640586014d90af9c351ccc7c094d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:42.021625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=625fa2d3f13a4350a9e2805f96c00d13, user_id=ce9e640586014d90af9c351ccc7c094d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:42.024546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:42.038521 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 941/1873] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:41 2022] PUT /identity/v3/projects/625fa2d3f13a4350a9e2805f96c00d13/users/ce9e640586014d90af9c351ccc7c094d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:42.053463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a68b-fd42-46b0-8a4c-67630c2843dd None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:42.053774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a68b-fd42-46b0-8a4c-67630c2843dd None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:42.054030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a68b-fd42-46b0-8a4c-67630c2843dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:42.094725 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-43d4a68b-fd42-46b0-8a4c-67630c2843dd None None] MFA Rules not processed for user `ce9e640586014d90af9c351ccc7c094d`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:42.098680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-43d4a68b-fd42-46b0-8a4c-67630c2843dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:42.113252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:42.244897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 933/1874] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:42.675175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:42.678493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:42.755762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c191d5-10d0-487d-9cc5-34620505c56b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:42.760360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:42.761194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:42.761665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:42.762401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:42.775056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:42.776663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:42.788486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 942/1875] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:43.205764 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a68b-fd42-46b0-8a4c-67630c2843dd None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:43.218837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a68b-fd42-46b0-8a4c-67630c2843dd None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:43.223022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:43.223640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:43.223984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:43.224299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:43.234097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:43.335216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:43.336790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:43.349883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 934/1876] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:43.626236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:44.187414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:44.197295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff652e1f-9269-430a-98ae-b8d461214850 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:44.201037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83d224-d200-4195-abf2-0107cfd94b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:44.201642 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d224-d200-4195-abf2-0107cfd94b7f service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:44.201915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d224-d200-4195-abf2-0107cfd94b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:44.202179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d224-d200-4195-abf2-0107cfd94b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:44.266068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83d224-d200-4195-abf2-0107cfd94b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:44.267507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83d224-d200-4195-abf2-0107cfd94b7f service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:44.277164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 943/1877] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:44.329307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:44.554863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:44.567330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:44.675286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a3ebe-b715-489a-92ef-f78e647b452a service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:44.686048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:44.686706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:44.686987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:44.687259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] PATH_INFO: `/v3/users/49818aa981cc4fbc8d6808fbe1065d6f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:44.688483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49818aa981cc4fbc8d6808fbe1065d6f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:44.690548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:44.991406 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49818aa981cc4fbc8d6808fbe1065d6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.012127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49818aa981cc4fbc8d6808fbe1065d6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.014046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49818aa981cc4fbc8d6808fbe1065d6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.025620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49818aa981cc4fbc8d6808fbe1065d6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.026525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49818aa981cc4fbc8d6808fbe1065d6f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.183198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 935/1878] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:44 2022] DELETE /identity/v3/users/49818aa981cc4fbc8d6808fbe1065d6f => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:45.185548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:45.185836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:45.186116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:45.220078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:45.222935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:45.298232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 944/1879] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:45.309933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:45.311500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:45.361115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:45.364039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:45.385189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695eb33-d305-4f00-b39b-e16463896e59 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:45.398175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:45.399174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:45.399743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:45.400339 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] PATH_INFO: `/v3/users/17b82a229cce49ca9d682e844df697a6` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:45.402453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b82a229cce49ca9d682e844df697a6)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:45.411222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:45.414359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b7794-46aa-47e9-a8a8-13966440b6d5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:45.424039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca01e1-b606-4af1-a202-bf65c143998a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:45.428402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca01e1-b606-4af1-a202-bf65c143998a service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:45.428951 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca01e1-b606-4af1-a202-bf65c143998a service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:45.429433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca01e1-b606-4af1-a202-bf65c143998a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:45.452072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca01e1-b606-4af1-a202-bf65c143998a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:45.453066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca01e1-b606-4af1-a202-bf65c143998a service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:45.474107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 936/1880] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:45.649621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b82a229cce49ca9d682e844df697a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.654188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b82a229cce49ca9d682e844df697a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.784840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b82a229cce49ca9d682e844df697a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.785473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b82a229cce49ca9d682e844df697a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.795028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b82a229cce49ca9d682e844df697a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:45.829058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 945/1881] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:45 2022] DELETE /identity/v3/users/17b82a229cce49ca9d682e844df697a6 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:45.831286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:45.831676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:45.832085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:45.844392 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:45.859308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:45.860732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:45.935472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 937/1882] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:45.942325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:45.944104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:46.014133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b94773-86b8-4ae0-af15-a40a822adb95 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:46.014133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:46.014707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:46.015197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:46.015630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:46.019433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:46.023479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:46.049019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:46.054392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:46.056378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c1d03-20fe-420a-85a2-1e0859b831ed None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:46.068008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 946/1883] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:46.073540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:46.074430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:46.075125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:46.075775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] PATH_INFO: `/v3/projects/4b5e76f54e504414aa904a2760e1b123` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:46.081583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5e76f54e504414aa904a2760e1b123)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:46.100300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:46.120608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:46.273668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5e76f54e504414aa904a2760e1b123', 'request_id': 'req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:46.280453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5e76f54e504414aa904a2760e1b123', 'request_id': 'req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:46.289472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5e76f54e504414aa904a2760e1b123', 'request_id': 'req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:46.469843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] The token cache is being invalidate because project 4b5e76f54e504414aa904a2760e1b123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:46.470486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:46.472174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:46.478836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 938/1884] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:46 2022] DELETE /identity/v3/projects/4b5e76f54e504414aa904a2760e1b123 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:46.482922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:46.512254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443587ee-209f-40fa-a7d4-2e54eb6a1f5e service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:46.656663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:46.657342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:46.657990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:46.658264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] PATH_INFO: `/v3/users/d0177c16b13e4e7fa2b8367e4b476162` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:46.659899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0177c16b13e4e7fa2b8367e4b476162)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:46.665255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:46.692324 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:46.693778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:46.729153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc304b9f-27d8-4ec1-9cd5-0036e243ba6c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:46.742483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:46.744318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:46.744990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:46.745655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] PATH_INFO: `/v3/projects/91220a5b455a48a9b288514d8e01623e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:46.756042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=91220a5b455a48a9b288514d8e01623e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:46.758556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:47.027245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91220a5b455a48a9b288514d8e01623e', 'request_id': 'req-a7b89270-1f80-4598-bb8c-78ab0feebd6a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.028088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0177c16b13e4e7fa2b8367e4b476162'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.029001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0177c16b13e4e7fa2b8367e4b476162'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.058291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91220a5b455a48a9b288514d8e01623e', 'request_id': 'req-a7b89270-1f80-4598-bb8c-78ab0feebd6a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.065761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91220a5b455a48a9b288514d8e01623e', 'request_id': 'req-a7b89270-1f80-4598-bb8c-78ab0feebd6a'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.208952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0177c16b13e4e7fa2b8367e4b476162'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.210071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0177c16b13e4e7fa2b8367e4b476162'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.228964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0177c16b13e4e7fa2b8367e4b476162'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.256749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 947/1885] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:46 2022] DELETE /identity/v3/users/d0177c16b13e4e7fa2b8367e4b476162 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:47.357382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] The token cache is being invalidate because project 91220a5b455a48a9b288514d8e01623e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:47.357963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.358890 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:47.361559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 939/1886] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:46 2022] DELETE /identity/v3/projects/91220a5b455a48a9b288514d8e01623e => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:47.366316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:47.368921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:47.513398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6aa6-33a7-4b37-bd28-fe9fa4ce101c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:47.519664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:47.520439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:47.520999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:47.532416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] PATH_INFO: `/v3/users/3af3a7e00814427696255707d9ed80f8` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:47.536077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af3a7e00814427696255707d9ed80f8)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:47.540766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:47.737653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:48.043673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:48.045148 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:48.083606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af3a7e00814427696255707d9ed80f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.083606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af3a7e00814427696255707d9ed80f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.148245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b89270-1f80-4598-bb8c-78ab0feebd6a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:48.154818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:48.155550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:48.155932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:48.156275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] PATH_INFO: `/v3/projects/4d93751d2cb14bdfb172c35acf9d9d69` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:48.157478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d93751d2cb14bdfb172c35acf9d9d69)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:48.159675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:48.280749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af3a7e00814427696255707d9ed80f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.282271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af3a7e00814427696255707d9ed80f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.294795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af3a7e00814427696255707d9ed80f8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.323589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 948/1887] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:47 2022] DELETE /identity/v3/users/3af3a7e00814427696255707d9ed80f8 => generated 0 bytes in 959 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:48.404772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d93751d2cb14bdfb172c35acf9d9d69', 'request_id': 'req-7487c329-8c51-4d9e-9175-201cd69ed1f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.412257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d93751d2cb14bdfb172c35acf9d9d69', 'request_id': 'req-7487c329-8c51-4d9e-9175-201cd69ed1f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.420399 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d93751d2cb14bdfb172c35acf9d9d69', 'request_id': 'req-7487c329-8c51-4d9e-9175-201cd69ed1f0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.629872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] The token cache is being invalidate because project 4d93751d2cb14bdfb172c35acf9d9d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:48.632240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.637871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:48.645378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 940/1888] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:48 2022] DELETE /identity/v3/projects/4d93751d2cb14bdfb172c35acf9d9d69 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:48.649312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:48.651930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:48.861574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a81e8-1397-4f67-96c9-ebd9bb5e36f5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:48.957682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:48.958338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:48.958618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:48.958892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] PATH_INFO: `/v3/users/0cd7d6c2be8a4852be716a3bc7312885` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:48.960052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd7d6c2be8a4852be716a3bc7312885)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:48.962687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:49.021856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:49.025682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:49.052659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487c329-8c51-4d9e-9175-201cd69ed1f0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:49.076161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:49.076161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:49.076161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:49.076161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] PATH_INFO: `/v3/projects/8ae1668b65d0401e9e057f3c9d0252bc` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:49.077029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae1668b65d0401e9e057f3c9d0252bc)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:49.078759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:49.271022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd7d6c2be8a4852be716a3bc7312885'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.272122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd7d6c2be8a4852be716a3bc7312885'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.329849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae1668b65d0401e9e057f3c9d0252bc', 'request_id': 'req-c04b96aa-c179-49c7-afe4-fce007c759ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.339169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd7d6c2be8a4852be716a3bc7312885'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.340198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd7d6c2be8a4852be716a3bc7312885'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.350311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae1668b65d0401e9e057f3c9d0252bc', 'request_id': 'req-c04b96aa-c179-49c7-afe4-fce007c759ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.353141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8afc796d-bacf-4c48-9190-491c2ac92c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd7d6c2be8a4852be716a3bc7312885'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.364936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae1668b65d0401e9e057f3c9d0252bc', 'request_id': 'req-c04b96aa-c179-49c7-afe4-fce007c759ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.386262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:49.393300 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 949/1889] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:48 2022] DELETE /identity/v3/users/0cd7d6c2be8a4852be716a3bc7312885 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:49.458682 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] The token cache is being invalidate because project 8ae1668b65d0401e9e057f3c9d0252bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:49.459220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.466781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c04b96aa-c179-49c7-afe4-fce007c759ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:49.466781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 941/1890] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:49 2022] DELETE /identity/v3/projects/8ae1668b65d0401e9e057f3c9d0252bc => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:49.470593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2804e-982a-4e6b-b877-de4149889c06 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:49.471084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2804e-982a-4e6b-b877-de4149889c06 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:49.473490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2804e-982a-4e6b-b877-de4149889c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:49.512327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-74e2804e-982a-4e6b-b877-de4149889c06 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:49.514844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-74e2804e-982a-4e6b-b877-de4149889c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:49.623505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 950/1891] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:49.632298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4885ce-a678-4428-8233-cd1fea740298 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:49.633006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4885ce-a678-4428-8233-cd1fea740298 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:49.633608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4885ce-a678-4428-8233-cd1fea740298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:49.667443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-fb4885ce-a678-4428-8233-cd1fea740298 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:49.668704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fb4885ce-a678-4428-8233-cd1fea740298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:49.699460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2804e-982a-4e6b-b877-de4149889c06 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:49.700720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 942/1892] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:49.710259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2804e-982a-4e6b-b877-de4149889c06 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:49.726917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:49.728147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:49.728696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:49.728977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:49.729819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:49.730126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:49.731806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:49.760004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4885ce-a678-4428-8233-cd1fea740298 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:49.766025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 951/1893] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:49.777229 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4885ce-a678-4428-8233-cd1fea740298 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:49.786085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:49.788167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:49.788167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:49.788167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:49.788940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:49.792383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:49.924972 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 943/1894] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:49 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:11:49.929331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:49.940962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029705e-303c-4da7-95f6-651fc3f94568 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.003417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.004163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.004535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.004909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.005734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:50.007892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:50.059456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.061270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.074773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5141332-9db0-469e-a242-b4bde1908996 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.079575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.080239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.080519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.080786 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] PATH_INFO: `/v3/projects/f9f46f2db2c94c949ec576323f25fe5a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.084013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f46f2db2c94c949ec576323f25fe5a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:50.084013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:50.117518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 952/1895] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:49 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:50.117682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.130607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303a7e-72bd-4586-9ae3-9fc46f77ad98 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.135893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.136491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.136767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.137034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.137836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:50.139453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:50.154536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 953/1896] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:50.159604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.171881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cd1bc-7e7d-4b7d-9469-eef46aaa5be7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.177190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.181111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.181111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.181111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.181111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:50.181970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:50.193427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 954/1897] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:50.200140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.232338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff32ca-783a-48c2-a142-93d6922ca0c1 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.242722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f46f2db2c94c949ec576323f25fe5a', 'request_id': 'req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.259247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f46f2db2c94c949ec576323f25fe5a', 'request_id': 'req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.273972 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f46f2db2c94c949ec576323f25fe5a', 'request_id': 'req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.311369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a91b6a-f606-4745-8e62-b17806fce051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.312066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91b6a-f606-4745-8e62-b17806fce051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.312346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91b6a-f606-4745-8e62-b17806fce051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.312622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91b6a-f606-4745-8e62-b17806fce051 admin admin] PATH_INFO: `/v3/projects/af1980f4ecd345f69d9fb776b9098732/users/ecf82c62238e488a89863520a4359257/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.326999 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a91b6a-f606-4745-8e62-b17806fce051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1980f4ecd345f69d9fb776b9098732, user_id=ecf82c62238e488a89863520a4359257, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:50.329746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a91b6a-f606-4745-8e62-b17806fce051 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:50.475910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] The token cache is being invalidate because project f9f46f2db2c94c949ec576323f25fe5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:50.488049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.488292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 955/1898] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:50 2022] PUT /identity/v3/projects/af1980f4ecd345f69d9fb776b9098732/users/ecf82c62238e488a89863520a4359257/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:50.488450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.488450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 944/1899] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:50 2022] DELETE /identity/v3/projects/f9f46f2db2c94c949ec576323f25fe5a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:50.493505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.494107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21750f5-b736-4398-9ec4-9be1cbe63cc4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.494457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21750f5-b736-4398-9ec4-9be1cbe63cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.494868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21750f5-b736-4398-9ec4-9be1cbe63cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.495185 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.509882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7e597-21c9-43f5-a3f0-f45216e75ef3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.515916 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e21750f5-b736-4398-9ec4-9be1cbe63cc4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:50.517318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e21750f5-b736-4398-9ec4-9be1cbe63cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.574877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.575576 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.575895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.576164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] PATH_INFO: `/v3/users/3f472eebdc9047bcbb32eb511cf5dd80` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.577302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f472eebdc9047bcbb32eb511cf5dd80)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:50.579215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:50.583611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 956/1900] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:50.593714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12fb4a-ef37-4495-a67d-f3526292ca6d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.593714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12fb4a-ef37-4495-a67d-f3526292ca6d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.593714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12fb4a-ef37-4495-a67d-f3526292ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.615760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7b12fb4a-ef37-4495-a67d-f3526292ca6d None None] MFA Rules not processed for user `ecf82c62238e488a89863520a4359257`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:50.617007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7b12fb4a-ef37-4495-a67d-f3526292ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.682157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 957/1901] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:50.683701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.683946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.684218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.703471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:50.704674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.713217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 958/1902] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:50.714083 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.715310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.726480 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd300858-8e2b-4d4a-a49d-4a5ea20fbe6e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.731617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.732220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.732498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.732770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.733405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:50.735281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:50.821596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f472eebdc9047bcbb32eb511cf5dd80'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.833712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f472eebdc9047bcbb32eb511cf5dd80'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.834369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f472eebdc9047bcbb32eb511cf5dd80'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.841665 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f472eebdc9047bcbb32eb511cf5dd80'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.842309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f472eebdc9047bcbb32eb511cf5dd80'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:50.877641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 959/1903] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:50 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:50.879219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.881857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.948063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b3f93-9d8a-4fe6-93fd-e34633549582 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.949312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 945/1904] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:50 2022] DELETE /identity/v3/users/3f472eebdc9047bcbb32eb511cf5dd80 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:50.950134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.953373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.011237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce29e9c-7e8d-4374-9323-12e85e571ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.011886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce29e9c-7e8d-4374-9323-12e85e571ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.012161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce29e9c-7e8d-4374-9323-12e85e571ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.012432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce29e9c-7e8d-4374-9323-12e85e571ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.013061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce29e9c-7e8d-4374-9323-12e85e571ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:51.013513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce29e9c-7e8d-4374-9323-12e85e571ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.015128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce29e9c-7e8d-4374-9323-12e85e571ccf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.019935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd215a2d-8834-4b20-85f6-d941e68d004f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.024900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.028722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.028722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.028722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.028722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.028722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.030765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 960/1905] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:11:51.035938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c09128-5b34-4f96-9346-64d7b4267b64 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.035938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c09128-5b34-4f96-9346-64d7b4267b64 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.035938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c09128-5b34-4f96-9346-64d7b4267b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.048706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:51.053506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a1c09128-5b34-4f96-9346-64d7b4267b64 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:51.057394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a1c09128-5b34-4f96-9346-64d7b4267b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.063622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 961/1906] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:51.064494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c09128-5b34-4f96-9346-64d7b4267b64 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.080394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c09128-5b34-4f96-9346-64d7b4267b64 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.086854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.086854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.086854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.086854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.087337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.087337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.151000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 946/1907] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:50 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:51.154044 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.154044 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.181466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 962/1908] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:51 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:51.182401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.211418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae36e-d38f-43b5-89b2-c1f062ece388 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.245629 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a132e-dfb5-4086-bf8c-dfebbc9da461 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.250554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.251518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.251518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.251708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.252598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.254189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.264618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 963/1909] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:51.264815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.265436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.265719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.265992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.269352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.278393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.279891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.283612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffca851-5934-4c49-a6d1-3647f9eb58da admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.288471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.289082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.289366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.289638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.291077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.291345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 947/1910] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:51.292311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.293185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.301935 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c861f-f5c3-473b-95c9-96f1672d4f86 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.308915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.308915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.309120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.309285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] PATH_INFO: `/v3/projects/099d53dd156349469cb91ae621130be7/users/802173c306604723a21e030c65ae1e52/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.325574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=099d53dd156349469cb91ae621130be7, user_id=802173c306604723a21e030c65ae1e52, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.328244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.402918 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 964/1911] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:51 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:51.407115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.418609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461ed9b-9f1a-4cb2-b4fb-2452777b1e71 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.423134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.423724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.424036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.424307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.424937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.426514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.439407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 965/1912] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:51.442498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 948/1913] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:51 2022] PUT /identity/v3/projects/099d53dd156349469cb91ae621130be7/users/802173c306604723a21e030c65ae1e52/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:51.450279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.450410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.462289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd158dae-3d2a-4d16-af51-1e0ce1e5144a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.513069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694fdfd-7220-4625-9cfb-23b1a8ea4838 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.517958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.518086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.518669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.518780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.518868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.518942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.519126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.519229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.519763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.519886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.521694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.521806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.533593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 966/1914] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:11:51.536980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 949/1915] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:51.541556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.541959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.554235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2087fa-a585-4e77-80fc-6ec92919ed9b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.554905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a0dee-2533-457d-b1c1-e70f36fea5df admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.560925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32baa5-605d-4ccc-8d40-dfe33f46396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.561092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9ec8d-0df3-4bce-a933-478ca495d66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.561748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32baa5-605d-4ccc-8d40-dfe33f46396a admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.561886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9ec8d-0df3-4bce-a933-478ca495d66d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.562077 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9ec8d-0df3-4bce-a933-478ca495d66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.562182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32baa5-605d-4ccc-8d40-dfe33f46396a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.562353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9ec8d-0df3-4bce-a933-478ca495d66d admin admin] PATH_INFO: `/v3/projects/1fd929a4f1ec46ce98f73462f98ba453/users/a1a63a726a8349a1a382094b6410706c/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.562547 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32baa5-605d-4ccc-8d40-dfe33f46396a admin admin] PATH_INFO: `/v3/projects/099d53dd156349469cb91ae621130be7/users/802173c306604723a21e030c65ae1e52/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.565201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32baa5-605d-4ccc-8d40-dfe33f46396a admin admin] RBAC: Authorizing `identity:create_grant(project_id=099d53dd156349469cb91ae621130be7, user_id=802173c306604723a21e030c65ae1e52, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.568286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32baa5-605d-4ccc-8d40-dfe33f46396a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.581004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9ec8d-0df3-4bce-a933-478ca495d66d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd929a4f1ec46ce98f73462f98ba453, user_id=a1a63a726a8349a1a382094b6410706c, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.581900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 967/1916] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:51 2022] PUT /identity/v3/projects/099d53dd156349469cb91ae621130be7/users/802173c306604723a21e030c65ae1e52/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:51.583619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9ec8d-0df3-4bce-a933-478ca495d66d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.594487 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.594909 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.595190 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.630591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] MFA Rules not processed for user `802173c306604723a21e030c65ae1e52`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:51.631819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.677502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 950/1917] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:51 2022] PUT /identity/v3/projects/1fd929a4f1ec46ce98f73462f98ba453/users/a1a63a726a8349a1a382094b6410706c/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:51.689181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.689521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.689797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.702855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 968/1918] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:51.720773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.722209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] MFA Rules not processed for user `a1a63a726a8349a1a382094b6410706c`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:51.722345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.723443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.781974 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0b66a-6ef6-4b41-b327-1de2694c0dc7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.787626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 951/1919] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:51.798329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.799981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.844921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.844921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.844921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.844921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.870645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee89e02-1cc2-46a8-bbf3-e5b15410d685 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.871598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:51.875445 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.876811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.876811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.876811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] PATH_INFO: `/v3/projects/c63543222fe8446dbad0a95ac6ea313a` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.878131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63543222fe8446dbad0a95ac6ea313a)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.880107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.900492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.907935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.912628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 969/1920] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:51.913371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:51.915005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.925993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bebc5-3f85-4d90-921d-a00642d99c3f service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:51.929723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:51.930325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:51.930637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:51.930913 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:51.939181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.940646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.949825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 970/1921] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:52.007666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63543222fe8446dbad0a95ac6ea313a', 'request_id': 'req-99a55527-f639-4d4e-8589-ef88f3aeb860'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.013867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63543222fe8446dbad0a95ac6ea313a', 'request_id': 'req-99a55527-f639-4d4e-8589-ef88f3aeb860'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.020691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63543222fe8446dbad0a95ac6ea313a', 'request_id': 'req-99a55527-f639-4d4e-8589-ef88f3aeb860'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.156793 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] The token cache is being invalidate because project c63543222fe8446dbad0a95ac6ea313a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:52.157441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.157719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.164543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 952/1922] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:51 2022] DELETE /identity/v3/projects/c63543222fe8446dbad0a95ac6ea313a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:52.164732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:52.174997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276341b8-e745-4444-8920-26d0693427a6 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:52.236671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:52.237336 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:52.237597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:52.237871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] PATH_INFO: `/v3/users/cd1f4445774846debe33796a20402d65` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:52.238994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1f4445774846debe33796a20402d65)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:52.240970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:52.261782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:52.263393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:52.320491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a55527-f639-4d4e-8589-ef88f3aeb860 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:52.325239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:52.325843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:52.326120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:52.326477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:52.328474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:52.413502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:52.415012 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:52.425358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 953/1923] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:11:52.500114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1f4445774846debe33796a20402d65'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.500609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1f4445774846debe33796a20402d65'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.606918 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1f4445774846debe33796a20402d65'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.608042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1f4445774846debe33796a20402d65'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.627677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1f4445774846debe33796a20402d65'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:52.668960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 971/1924] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:52 2022] DELETE /identity/v3/users/cd1f4445774846debe33796a20402d65 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:52.774926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:53.555241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:53.558637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:53.574772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa75c2-1895-43f1-b011-16b1f991a6bf service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:53.687754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:53.689971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:53.690743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:53.691422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] PATH_INFO: `/v3/projects/7e39058b07d94acc86b50dc1fdd7c64f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:53.694759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e39058b07d94acc86b50dc1fdd7c64f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:53.699002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:53.865207 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e39058b07d94acc86b50dc1fdd7c64f', 'request_id': 'req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:53.875085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e39058b07d94acc86b50dc1fdd7c64f', 'request_id': 'req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:53.885028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e39058b07d94acc86b50dc1fdd7c64f', 'request_id': 'req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:53.971304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] The token cache is being invalidate because project 7e39058b07d94acc86b50dc1fdd7c64f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:53.972140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:53.973264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:53.976345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 954/1925] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:53 2022] DELETE /identity/v3/projects/7e39058b07d94acc86b50dc1fdd7c64f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:53.982013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:53.987601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:54.032440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51244db7-7e17-4812-9dc2-ac560021978c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:54.200429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:54.201414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:54.201879 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:54.202344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] PATH_INFO: `/v3/users/bcf73b0932a743e3b764bf52e01e4e21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:54.204319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf73b0932a743e3b764bf52e01e4e21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:54.207995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:54.265720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:54.489012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf73b0932a743e3b764bf52e01e4e21'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:54.492115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf73b0932a743e3b764bf52e01e4e21'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:54.572748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf73b0932a743e3b764bf52e01e4e21'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:54.573394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf73b0932a743e3b764bf52e01e4e21'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:54.585360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-23a0bafc-9a73-4a4d-ac5e-f13668c287b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf73b0932a743e3b764bf52e01e4e21'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:54.622291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 972/1926] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:53 2022] DELETE /identity/v3/users/bcf73b0932a743e3b764bf52e01e4e21 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:54.638612 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:55.221068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:55.223060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:55.254543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065b0eb-356c-410d-b3c2-0cdb3e451fe3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:55.331564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:55.332304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:55.332563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:55.333036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] PATH_INFO: `/v3/projects/268e09dd0c264c56b37b7248753de41e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:55.334476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268e09dd0c264c56b37b7248753de41e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:55.337393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:55.548538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268e09dd0c264c56b37b7248753de41e', 'request_id': 'req-0c6c8037-83ab-4f87-84c0-14d37364a893'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:55.558006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268e09dd0c264c56b37b7248753de41e', 'request_id': 'req-0c6c8037-83ab-4f87-84c0-14d37364a893'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:55.576617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268e09dd0c264c56b37b7248753de41e', 'request_id': 'req-0c6c8037-83ab-4f87-84c0-14d37364a893'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:55.699327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] The token cache is being invalidate because project 268e09dd0c264c56b37b7248753de41e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:55.700542 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:55.701395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-0c6c8037-83ab-4f87-84c0-14d37364a893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:55.703287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 955/1927] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:55 2022] DELETE /identity/v3/projects/268e09dd0c264c56b37b7248753de41e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:55.709899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed499b7-1cab-492e-ad0e-30e52d00034b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:55.710420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed499b7-1cab-492e-ad0e-30e52d00034b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:55.710577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed499b7-1cab-492e-ad0e-30e52d00034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:55.747291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0ed499b7-1cab-492e-ad0e-30e52d00034b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:55.748629 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0ed499b7-1cab-492e-ad0e-30e52d00034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:55.877446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 973/1928] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:55.885740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c918b-3e90-4afc-ba31-89e6707738e6 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:55.886038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c918b-3e90-4afc-ba31-89e6707738e6 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:55.886317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c918b-3e90-4afc-ba31-89e6707738e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:55.908776 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ce3c918b-3e90-4afc-ba31-89e6707738e6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:55.910390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ce3c918b-3e90-4afc-ba31-89e6707738e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:55.925743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 956/1929] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:55.938950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed499b7-1cab-492e-ad0e-30e52d00034b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:55.952472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed499b7-1cab-492e-ad0e-30e52d00034b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:55.959014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:55.959641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:55.959958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:55.960388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:55.960926 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:11:55.961232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:55.963108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:55.980028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 974/1930] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:11:55.989461 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c918b-3e90-4afc-ba31-89e6707738e6 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:56.005944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c918b-3e90-4afc-ba31-89e6707738e6 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:56.046743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:56.047370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.047652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.047956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.049256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:56.050541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:56.234782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 957/1931] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:55 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:56.247006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:56.248632 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:56.268625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ac55f-3946-4e8e-9138-7a2057402af3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:56.367345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:56.369412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.376135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.380794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.381508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:56.388582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:56.430149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:56.513427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:56.516385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:56.538517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0bee-aeaf-4790-aac1-6354583f91a2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:56.545710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:56.546619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.547991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.548753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] PATH_INFO: `/v3/users/a60b6110604e454e97f1382035b91e5b` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.550637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60b6110604e454e97f1382035b91e5b)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:56.553435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:56.578098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 975/1932] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:56 2022] POST /identity/v3/users => generated 291 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:56.584252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:56.608324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9ac8f-c2ea-49bb-8682-8501a81b1984 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:56.613243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:56.614617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.615087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.615364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.616505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:56.619058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:56.635090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 976/1933] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:56.638951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:56.656340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067927c-a8de-4aad-b4c5-e11b4c02dd7e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:56.662460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:56.663619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.664109 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.664637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.665897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:56.667587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:56.679047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 977/1934] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:56.684331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:56.694702 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc310ea0-d0e5-45fc-aa16-0e8f9dfeb763 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:56.700113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841c1a-a837-4ba3-8bdd-124c7bd1238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:56.700782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841c1a-a837-4ba3-8bdd-124c7bd1238b admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.701123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841c1a-a837-4ba3-8bdd-124c7bd1238b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.701400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841c1a-a837-4ba3-8bdd-124c7bd1238b admin admin] PATH_INFO: `/v3/projects/7b550446098640a3a321e8bb5ea068cd/users/1acefed3c2914b63a8d9fd07c475bf85/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.718947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25841c1a-a837-4ba3-8bdd-124c7bd1238b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b550446098640a3a321e8bb5ea068cd, user_id=1acefed3c2914b63a8d9fd07c475bf85, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:56.722374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25841c1a-a837-4ba3-8bdd-124c7bd1238b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:56.828431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 978/1935] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:56 2022] PUT /identity/v3/projects/7b550446098640a3a321e8bb5ea068cd/users/1acefed3c2914b63a8d9fd07c475bf85/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:56.845885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.845885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.846115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60b6110604e454e97f1382035b91e5b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:56.846526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.875642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60b6110604e454e97f1382035b91e5b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:56.876566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60b6110604e454e97f1382035b91e5b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:56.912550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60b6110604e454e97f1382035b91e5b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:56.921545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-9df9aa69-14cc-417d-8264-d0b4ecca1880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60b6110604e454e97f1382035b91e5b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:56.927952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] MFA Rules not processed for user `1acefed3c2914b63a8d9fd07c475bf85`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:56.929735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:57.016095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 979/1936] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:57.031586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:57.034842 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:57.115446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 958/1937] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:56 2022] DELETE /identity/v3/users/a60b6110604e454e97f1382035b91e5b => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:57.121853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.122688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.123310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.141304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82140d-9e2b-4c4d-841d-f21acafc3c29 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:57.148756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:57.150339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:57.223894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:57.224591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.224954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.225309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.226019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:57.231298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:57.238209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 959/1938] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:57.253195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:57.262098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:57.326479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87226e9-26e1-45f1-8412-7473edf2bb2d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:57.333136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:57.349156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.349156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.349156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.356812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:57.358467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:57.378094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 960/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:57.422914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 980/1940] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:11:57 2022] POST /identity/v3/projects => generated 381 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:57.427732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:57.445083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b8f8-1aec-43ce-8fe4-dd8d6b091e72 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:57.560626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:57.561520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.562020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.562359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.563150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:57.565144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:57.738141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 961/1941] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:11:57 2022] POST /identity/v3/users => generated 297 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:11:57.739481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:57.753978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5f7fe-e40c-46bc-8578-24dc1b09775a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:57.759635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:57.760466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.760862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.761231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.762122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:57.764114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:57.793598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:57.821084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 981/1942] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:57.826830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb43df7-321e-41df-8381-ce510d811e39 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:57.833466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:57.834298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.834734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.835150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] PATH_INFO: `/v3/projects/ea6c284ac7324554996f7af605563fc7/users/ba3136fbd87e464d9a542ae88b41e9fa/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.858090 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6c284ac7324554996f7af605563fc7, user_id=ba3136fbd87e464d9a542ae88b41e9fa, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:57.862414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:57.879346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:58.009929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:58.010961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 962/1943] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:57 2022] PUT /identity/v3/projects/ea6c284ac7324554996f7af605563fc7/users/ba3136fbd87e464d9a542ae88b41e9fa/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:58.012292 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.015631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:58.032726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73524387-47a1-440d-bac6-e57b6e2d5d12 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:58.157091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488789a-fc5b-4db5-8a6e-09e50e6682c3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:58.160616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:58.165583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.165583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.165583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] PATH_INFO: `/v3/projects/decf5d1a75104540a5955b930b990275` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.165583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=decf5d1a75104540a5955b930b990275)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:58.166120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:58.166851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.167238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.167558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:58.167671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.168658 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:58.170707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:58.188351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 963/1944] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:11:58.193842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:58.208293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81236e-0ebc-4850-af91-98331dd1b3a1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:58.219361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe84359-4d63-444c-a63a-e5592dc24c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:58.221256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84359-4d63-444c-a63a-e5592dc24c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.221541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84359-4d63-444c-a63a-e5592dc24c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.221846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84359-4d63-444c-a63a-e5592dc24c45 admin admin] PATH_INFO: `/v3/projects/ea6c284ac7324554996f7af605563fc7/users/ba3136fbd87e464d9a542ae88b41e9fa/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.224581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe84359-4d63-444c-a63a-e5592dc24c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6c284ac7324554996f7af605563fc7, user_id=ba3136fbd87e464d9a542ae88b41e9fa, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:58.228131 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe84359-4d63-444c-a63a-e5592dc24c45 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:58.243078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 964/1945] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:11:58 2022] PUT /identity/v3/projects/ea6c284ac7324554996f7af605563fc7/users/ba3136fbd87e464d9a542ae88b41e9fa/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:58.260766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.261275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.261561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.294265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] MFA Rules not processed for user `ba3136fbd87e464d9a542ae88b41e9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:58.295965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.417452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 965/1946] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:58.418416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:58.423192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.435728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'decf5d1a75104540a5955b930b990275', 'request_id': 'req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.443274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'decf5d1a75104540a5955b930b990275', 'request_id': 'req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.452677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'decf5d1a75104540a5955b930b990275', 'request_id': 'req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.506648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb24ac1-32ab-4d3b-b95c-60a8c6cb3ba3 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:58.581461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] The token cache is being invalidate because project decf5d1a75104540a5955b930b990275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:11:58.582186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.583109 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.588751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 982/1947] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:11:58 2022] DELETE /identity/v3/projects/decf5d1a75104540a5955b930b990275 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:11:58.588751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:58.589246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.602347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:58.604002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.604201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.604443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] PATH_INFO: `/v3/users/ecf82c62238e488a89863520a4359257` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.606303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf82c62238e488a89863520a4359257)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:58.607754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:58.670801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4161a-3eee-4b1e-b1dd-ab63742aac77 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:58.678927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:58.679553 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.679861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.680137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.682055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.752906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:58.754385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:58.764960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 983/1948] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:58.765828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:58.767116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.780305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04780afd-81ac-45ef-9ad4-7ae1d0351cb4 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:58.785350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:58.785975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.786255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.786536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] PATH_INFO: `/v3/users/6a2abb95bb3041f49a52dd21ff770c91` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.787658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a2abb95bb3041f49a52dd21ff770c91)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:58.789733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:58.819988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:58.912414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf82c62238e488a89863520a4359257'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.928467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf82c62238e488a89863520a4359257'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.929306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf82c62238e488a89863520a4359257'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.937243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf82c62238e488a89863520a4359257'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:58.938049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf82c62238e488a89863520a4359257'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.077361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 966/1949] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:58 2022] DELETE /identity/v3/users/ecf82c62238e488a89863520a4359257 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:59.079015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:59.083227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.126044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a2abb95bb3041f49a52dd21ff770c91'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.126823 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a2abb95bb3041f49a52dd21ff770c91'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.168655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd5509-e184-40b1-a12d-7c00c12974ba admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:59.173447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:59.174084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.174361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.174633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] PATH_INFO: `/v3/users/e770e3cdf5de4d25bdfc614684c408ae` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.175766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e770e3cdf5de4d25bdfc614684c408ae)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:59.177806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:59.235130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a2abb95bb3041f49a52dd21ff770c91'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.236121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a2abb95bb3041f49a52dd21ff770c91'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.243827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a2abb95bb3041f49a52dd21ff770c91'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.263967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 984/1950] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:58 2022] DELETE /identity/v3/users/6a2abb95bb3041f49a52dd21ff770c91 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:59.264881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:59.266734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.342422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6000312-30c5-4458-8014-d34a0a99698f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:59.346456 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95850-57e0-45c3-9a46-7a22f07267b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:59.347081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95850-57e0-45c3-9a46-7a22f07267b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.347361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95850-57e0-45c3-9a46-7a22f07267b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.347639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95850-57e0-45c3-9a46-7a22f07267b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.350271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a9c95850-57e0-45c3-9a46-7a22f07267b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.429288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c95850-57e0-45c3-9a46-7a22f07267b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:59.430907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c95850-57e0-45c3-9a46-7a22f07267b4 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:59.440350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 985/1951] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:59.449983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.450321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.450601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.492283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e770e3cdf5de4d25bdfc614684c408ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.495638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:59.519641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.539488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e770e3cdf5de4d25bdfc614684c408ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.541366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e770e3cdf5de4d25bdfc614684c408ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.565391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e770e3cdf5de4d25bdfc614684c408ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.573064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ad95c2d3-42d4-463d-b8ca-4d0eb209a0c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e770e3cdf5de4d25bdfc614684c408ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:11:59.634309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 986/1952] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:59.645163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:59.648395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.726558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 967/1953] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:59 2022] DELETE /identity/v3/users/e770e3cdf5de4d25bdfc614684c408ae => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:11:59.731692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.732511 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.732791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.770040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f225c9-755b-4f07-8e47-9606a3af95cc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:59.774519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:59.778878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.889565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:59.895470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.895470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.895470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.923186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 968/1954] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:59.924236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:59.927213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.942119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184d153-dd63-47ac-8fd4-61b64241e92a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:59.956701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:59.956701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.956701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.956701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] PATH_INFO: `/v3/users/159d270564504fab86a7cee778a4dfd4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.957485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=159d270564504fab86a7cee778a4dfd4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:59.957485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:00.026403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:00.073678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:00.076522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:00.086801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 987/1955] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:12:00.088567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:00.089939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.123820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a45c1-04c8-49de-a5e5-0da124b42e6d service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:00.132584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:00.133262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:00.133524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:00.134652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] PATH_INFO: `/v3/projects/eba7a47186d14007be32666105436717` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:00.143555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba7a47186d14007be32666105436717)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:00.143555 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:00.302720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '159d270564504fab86a7cee778a4dfd4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.330541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba7a47186d14007be32666105436717', 'request_id': 'req-30eea336-c194-483c-8c48-56d2aedee165'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.337785 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba7a47186d14007be32666105436717', 'request_id': 'req-30eea336-c194-483c-8c48-56d2aedee165'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.342724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '159d270564504fab86a7cee778a4dfd4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.344512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '159d270564504fab86a7cee778a4dfd4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.345978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba7a47186d14007be32666105436717', 'request_id': 'req-30eea336-c194-483c-8c48-56d2aedee165'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.356122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '159d270564504fab86a7cee778a4dfd4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.356872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '159d270564504fab86a7cee778a4dfd4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.392039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:00.445610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] The token cache is being invalidate because project eba7a47186d14007be32666105436717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:00.445914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.447337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.448924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 988/1956] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:00 2022] DELETE /identity/v3/projects/eba7a47186d14007be32666105436717 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:00.454348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:00.458878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.501435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 969/1957] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:11:59 2022] DELETE /identity/v3/users/159d270564504fab86a7cee778a4dfd4 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:00.502920 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:00.508034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.615120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eea336-c194-483c-8c48-56d2aedee165 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:00.644544 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5b9f3-ed60-4f7e-8125-08ddb88c41f5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:00.657111 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:00.657773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:00.658054 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:00.658329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] PATH_INFO: `/v3/users/1acefed3c2914b63a8d9fd07c475bf85` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:00.660468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1acefed3c2914b63a8d9fd07c475bf85)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:00.668130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:00.689765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1436c83-fa6d-48ac-94c0-150921da3964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:00.691900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436c83-fa6d-48ac-94c0-150921da3964 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:00.692186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436c83-fa6d-48ac-94c0-150921da3964 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:00.692462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436c83-fa6d-48ac-94c0-150921da3964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:00.695993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f1436c83-fa6d-48ac-94c0-150921da3964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.745490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1436c83-fa6d-48ac-94c0-150921da3964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:00.747171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1436c83-fa6d-48ac-94c0-150921da3964 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:00.760609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 989/1958] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:00.762160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68f84-8d74-4248-b4f4-3e5ebd0a873c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:00.762445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68f84-8d74-4248-b4f4-3e5ebd0a873c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:00.762753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68f84-8d74-4248-b4f4-3e5ebd0a873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:00.792914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e6d68f84-8d74-4248-b4f4-3e5ebd0a873c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:00.794378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e6d68f84-8d74-4248-b4f4-3e5ebd0a873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.808094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 990/1959] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:00.809431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292c16e-d81b-4b94-982a-d331be075000 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:00.809798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292c16e-d81b-4b94-982a-d331be075000 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:00.810100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292c16e-d81b-4b94-982a-d331be075000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:00.833098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1292c16e-d81b-4b94-982a-d331be075000 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:00.834492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1292c16e-d81b-4b94-982a-d331be075000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.857950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 991/1960] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:00.861623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:00.862719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:00.863608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:00.872297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:00.918274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:00.918274 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.938756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 992/1961] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:00.942979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:00.944476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.958670 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474add5f-8ef3-4c7f-b0ff-74ce14d35d63 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:00.966898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1acefed3c2914b63a8d9fd07c475bf85'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.971319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:00.972001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:00.972280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:00.972558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] PATH_INFO: `/v3/projects/af1980f4ecd345f69d9fb776b9098732` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:00.973819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1980f4ecd345f69d9fb776b9098732)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:00.981681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:00.990775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1acefed3c2914b63a8d9fd07c475bf85'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:00.996145 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1acefed3c2914b63a8d9fd07c475bf85'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.003007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1acefed3c2914b63a8d9fd07c475bf85'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.005112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1acefed3c2914b63a8d9fd07c475bf85'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.143223 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 970/1962] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:00 2022] DELETE /identity/v3/users/1acefed3c2914b63a8d9fd07c475bf85 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:01.144332 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:01.151081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:01.151381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:01.170170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1980f4ecd345f69d9fb776b9098732', 'request_id': 'req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.176760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1980f4ecd345f69d9fb776b9098732', 'request_id': 'req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.184611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1980f4ecd345f69d9fb776b9098732', 'request_id': 'req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.279994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269d02-aa4b-494c-bb11-922ca9129a0e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:01.295159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] The token cache is being invalidate because project af1980f4ecd345f69d9fb776b9098732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:01.295728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.297036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.300720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 993/1963] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:00 2022] DELETE /identity/v3/projects/af1980f4ecd345f69d9fb776b9098732 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:01.303413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:01.309866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:01.427219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:01.427879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:01.428191 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:01.428469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:01.438058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:01.446127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f51fc-4659-42fb-b53e-6ca4b66e9e6b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:01.452101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f774cd-254c-4bdf-a677-8bfb15bc83e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:01.453087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f774cd-254c-4bdf-a677-8bfb15bc83e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:01.453375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f774cd-254c-4bdf-a677-8bfb15bc83e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:01.453650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f774cd-254c-4bdf-a677-8bfb15bc83e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:01.454310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f774cd-254c-4bdf-a677-8bfb15bc83e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:01.454617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f774cd-254c-4bdf-a677-8bfb15bc83e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:01.456467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f774cd-254c-4bdf-a677-8bfb15bc83e3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:01.469896 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:01.473741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 994/1964] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:01.474510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:01.474804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:01.475107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:01.481391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:01.498843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:01.500111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:01.511438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 995/1965] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:01.512141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:01.513463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:01.518463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 971/1966] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:01.519423 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:01.530218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8acda-4b0a-4b62-9161-92e22247f4d7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:01.538356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:01.539479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:01.539899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:01.540401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] PATH_INFO: `/v3/users/802173c306604723a21e030c65ae1e52` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:01.541791 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802173c306604723a21e030c65ae1e52)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:01.544145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:01.544802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6d5a7-f0bd-4cdf-b21c-ba33bade0813 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:01.558808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:01.559442 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:01.559726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:01.560050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:01.560755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:01.564522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:01.748167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 972/1967] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:01 2022] POST /identity/v3/projects => generated 359 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:01.752201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:01.755137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:01.854522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dae95-9153-4d49-9da5-fae36bd26174 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:01.856229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802173c306604723a21e030c65ae1e52'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.857029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802173c306604723a21e030c65ae1e52'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.864252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:01.875456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:01.875456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:01.875456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:01.983511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802173c306604723a21e030c65ae1e52'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.984141 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802173c306604723a21e030c65ae1e52'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:01.996961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802173c306604723a21e030c65ae1e52'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.003867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.006184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.019764 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 973/1968] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:02.021201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.022794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.064936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079a537-d30e-4bce-855a-c7713bb293dc service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:02.073661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:02.075327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.075621 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.076066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] PATH_INFO: `/v3/projects/1fc325a95bfd4a71b4b1e4fc107caa09` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.077510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 996/1969] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:01 2022] DELETE /identity/v3/users/802173c306604723a21e030c65ae1e52 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:02.078448 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.085675 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc325a95bfd4a71b4b1e4fc107caa09)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.088410 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.089128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.280520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0769b-868c-4f8c-9e2e-ba1bd0b54fe9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:02.281772 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc325a95bfd4a71b4b1e4fc107caa09', 'request_id': 'req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.294322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc325a95bfd4a71b4b1e4fc107caa09', 'request_id': 'req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.308447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc325a95bfd4a71b4b1e4fc107caa09', 'request_id': 'req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.395441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:02.396273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.396745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.397191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.397958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.401030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.493518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] The token cache is being invalidate because project 1fc325a95bfd4a71b4b1e4fc107caa09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:02.493979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.495608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.497891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 974/1970] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:02 2022] DELETE /identity/v3/projects/1fc325a95bfd4a71b4b1e4fc107caa09 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:02.502190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.505145 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.519363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 997/1971] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:02 2022] POST /identity/v3/users => generated 280 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:02.523489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.530991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.552518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0283c-95e7-461c-8bd2-2ecef83d5a8f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:02.558895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63189-ecd2-4279-969e-19b538966552 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:02.573860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:02.574548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:02.574894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.575319 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.576099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.576270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] PATH_INFO: `/v3/users/ce9e640586014d90af9c351ccc7c094d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.576683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.577153 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.578056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.578872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9e640586014d90af9c351ccc7c094d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.579970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.583404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.598542 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 998/1972] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:02.599772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.612673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:02.616298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fea17-aa83-4eff-9627-76fe7a635381 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:02.630425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:02.631204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.632490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.633710 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.635705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.638068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.660399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 999/1973] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:02.668849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.714351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f01ded-37aa-40bb-9a31-6c6346c6743a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:02.724285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:02.725175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.725457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.725858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] PATH_INFO: `/v3/projects/0146872607c54eb48673395210465233/users/ff3377e43eff4f66984b579e8b84eeca/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.744724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0146872607c54eb48673395210465233, user_id=ff3377e43eff4f66984b579e8b84eeca, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.749315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.919583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1000/1974] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:02 2022] PUT /identity/v3/projects/0146872607c54eb48673395210465233/users/ff3377e43eff4f66984b579e8b84eeca/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:02.923625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.928659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.948312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9e640586014d90af9c351ccc7c094d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.968787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9e640586014d90af9c351ccc7c094d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.977157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9e640586014d90af9c351ccc7c094d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.986988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9e640586014d90af9c351ccc7c094d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:02.988040 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9e640586014d90af9c351ccc7c094d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.069761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa92e03-c8f4-43ce-969a-c0e54c3dcd96 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:03.075690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:03.076370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:03.076648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:03.076920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] PATH_INFO: `/v3/projects/d72a205f40ae47289414abe2c2b4d3ad` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:03.078104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72a205f40ae47289414abe2c2b4d3ad)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:03.080056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:03.163566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 975/1975] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:02 2022] DELETE /identity/v3/users/ce9e640586014d90af9c351ccc7c094d => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:03.163566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:03.163566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:03.273370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e08e44-501b-4f14-b66f-3dbf0302578b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:03.279891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:03.283676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:03.283676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:03.283676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] PATH_INFO: `/v3/projects/099d53dd156349469cb91ae621130be7` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:03.283676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=099d53dd156349469cb91ae621130be7)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:03.285021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:03.315721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:03.343473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72a205f40ae47289414abe2c2b4d3ad', 'request_id': 'req-dc20e467-639d-4829-9adb-958bf3fe8021'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.349927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72a205f40ae47289414abe2c2b4d3ad', 'request_id': 'req-dc20e467-639d-4829-9adb-958bf3fe8021'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.379441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72a205f40ae47289414abe2c2b4d3ad', 'request_id': 'req-dc20e467-639d-4829-9adb-958bf3fe8021'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.496914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099d53dd156349469cb91ae621130be7', 'request_id': 'req-f95b7538-519c-4384-83b3-92e610028724'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.506263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099d53dd156349469cb91ae621130be7', 'request_id': 'req-f95b7538-519c-4384-83b3-92e610028724'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.515683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099d53dd156349469cb91ae621130be7', 'request_id': 'req-f95b7538-519c-4384-83b3-92e610028724'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.529006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] The token cache is being invalidate because project d72a205f40ae47289414abe2c2b4d3ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:03.529301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.530520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc20e467-639d-4829-9adb-958bf3fe8021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.532416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1001/1976] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:02 2022] DELETE /identity/v3/projects/d72a205f40ae47289414abe2c2b4d3ad => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:03.535506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:03.535915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:03.536191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:03.604481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] MFA Rules not processed for user `ff3377e43eff4f66984b579e8b84eeca`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:03.613757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:03.652283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] The token cache is being invalidate because project 099d53dd156349469cb91ae621130be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:03.653197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.654843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:03.657484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 976/1977] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:03 2022] DELETE /identity/v3/projects/099d53dd156349469cb91ae621130be7 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:03.658700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:03.660704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:03.711313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1002/1978] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:03.712978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:03.714775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:03.812010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7538-519c-4384-83b3-92e610028724 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:03.818279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:03.819064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:03.819453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:03.819869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] PATH_INFO: `/v3/projects/7b550446098640a3a321e8bb5ea068cd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:03.821130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b550446098640a3a321e8bb5ea068cd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:03.823842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:03.824177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0180be1-666b-48d1-b152-758d255d58f3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:03.832346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:03.833207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:03.833636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:03.833944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] PATH_INFO: `/v3/users/873a9e099149474690c4872149c5177e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:03.836738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] RBAC: Authorizing `identity:delete_user(user_id=873a9e099149474690c4872149c5177e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:03.838554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:04.559129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873a9e099149474690c4872149c5177e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.559890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873a9e099149474690c4872149c5177e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.560491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b550446098640a3a321e8bb5ea068cd', 'request_id': 'req-10a55a3d-9a10-465d-abc8-323ce28cee72'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.567027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b550446098640a3a321e8bb5ea068cd', 'request_id': 'req-10a55a3d-9a10-465d-abc8-323ce28cee72'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.575030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b550446098640a3a321e8bb5ea068cd', 'request_id': 'req-10a55a3d-9a10-465d-abc8-323ce28cee72'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.657762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873a9e099149474690c4872149c5177e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.658413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873a9e099149474690c4872149c5177e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.672430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-998cb749-bd8d-4b08-83ae-c02964b2927b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873a9e099149474690c4872149c5177e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.694736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1003/1979] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:03 2022] DELETE /identity/v3/users/873a9e099149474690c4872149c5177e => generated 0 bytes in 983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:04.696255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:04.697066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:04.697339 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:04.728601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:04.729914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:04.734242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] The token cache is being invalidate because project 7b550446098640a3a321e8bb5ea068cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:04.739143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.739143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:04.739143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 977/1980] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:03 2022] DELETE /identity/v3/projects/7b550446098640a3a321e8bb5ea068cd => generated 0 bytes in 1080 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:04.739143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:04.741974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:04.806331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:04.838976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a55a3d-9a10-465d-abc8-323ce28cee72 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:04.852370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1004/1981] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:04.853505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:04.854987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:04.870005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc435da-bd43-4ce1-ad89-597d9c6288c7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:04.875556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:04.876683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:04.879057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:04.879459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] PATH_INFO: `/v3/projects/625fa2d3f13a4350a9e2805f96c00d13` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:04.883576 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625fa2d3f13a4350a9e2805f96c00d13)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:04.887951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:04.914082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:04.916448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:04.917829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:04.918218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:04.920831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:04.989386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:04.990862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:05.002071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 978/1982] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:05.003420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:05.004910 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.036116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a5512-33c7-47d3-8317-3a30281a0fe0 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:05.056887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625fa2d3f13a4350a9e2805f96c00d13', 'request_id': 'req-c7710477-6b0f-472a-8a92-c7ff5344b368'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.065289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625fa2d3f13a4350a9e2805f96c00d13', 'request_id': 'req-c7710477-6b0f-472a-8a92-c7ff5344b368'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.078027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625fa2d3f13a4350a9e2805f96c00d13', 'request_id': 'req-c7710477-6b0f-472a-8a92-c7ff5344b368'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.117455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:05.138507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:05.138507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.138507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.138507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] PATH_INFO: `/v3/users/ba3136fbd87e464d9a542ae88b41e9fa` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.138981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3136fbd87e464d9a542ae88b41e9fa)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:05.140739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:05.168793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] The token cache is being invalidate because project 625fa2d3f13a4350a9e2805f96c00d13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:05.170425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.171726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c7710477-6b0f-472a-8a92-c7ff5344b368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.175482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1005/1983] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:04 2022] DELETE /identity/v3/projects/625fa2d3f13a4350a9e2805f96c00d13 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:05.178216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.179685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.180160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.220579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:05.222165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.236311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1006/1984] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:05.237593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:05.239236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.252301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014961-892d-41d8-8167-24af5ce13ea4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:05.260496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:05.271897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.271897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.271897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] PATH_INFO: `/v3/users/ff3377e43eff4f66984b579e8b84eeca` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.271897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3377e43eff4f66984b579e8b84eeca)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:05.271897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:05.428209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3136fbd87e464d9a542ae88b41e9fa'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.449715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3136fbd87e464d9a542ae88b41e9fa'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.451015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3136fbd87e464d9a542ae88b41e9fa'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.457741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3136fbd87e464d9a542ae88b41e9fa'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.458483 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-d8efe6d5-dc59-4986-8a2a-7b594aa3723a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3136fbd87e464d9a542ae88b41e9fa'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.500420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3377e43eff4f66984b579e8b84eeca'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.501399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3377e43eff4f66984b579e8b84eeca'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.644713 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 979/1985] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:05 2022] DELETE /identity/v3/users/ba3136fbd87e464d9a542ae88b41e9fa => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:05.644713 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.644713 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.645438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.665374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:05.666605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.732864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 980/1986] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:05.733805 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:05.735081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.740406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3377e43eff4f66984b579e8b84eeca'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.741396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3377e43eff4f66984b579e8b84eeca'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.748603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b397-45d5-4148-abe5-945944c26a9b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:05.749297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3377e43eff4f66984b579e8b84eeca'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:05.753765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4880635-902a-480f-b6ca-d50442cd4cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:05.754374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880635-902a-480f-b6ca-d50442cd4cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.754654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880635-902a-480f-b6ca-d50442cd4cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.754929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880635-902a-480f-b6ca-d50442cd4cc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.755626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4880635-902a-480f-b6ca-d50442cd4cc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:05.755962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4880635-902a-480f-b6ca-d50442cd4cc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:05.757585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4880635-902a-480f-b6ca-d50442cd4cc5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:05.767858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1007/1987] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:05 2022] DELETE /identity/v3/users/ff3377e43eff4f66984b579e8b84eeca => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:05.768672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:05.769979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.771831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 981/1988] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:05.771981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae6211-093d-48f5-8bda-77620af6948e None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.772374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae6211-093d-48f5-8bda-77620af6948e None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.772697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae6211-093d-48f5-8bda-77620af6948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.792873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7dae6211-093d-48f5-8bda-77620af6948e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:05.794097 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7dae6211-093d-48f5-8bda-77620af6948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.838148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d022-d626-418d-a34c-1bd44df02d46 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:05.842788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:05.843506 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.844037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.844411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] PATH_INFO: `/v3/projects/1b07ff4612e7420d84fe8d0b75e04c6b` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.845624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b07ff4612e7420d84fe8d0b75e04c6b)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:05.847621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:05.856089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 982/1989] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:05.857170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.857473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.857750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.878472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:05.879777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.888363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 983/1990] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:05.889066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:05.892431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.904670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe81648-c172-4867-b604-c8631afbf8e5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:05.909700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:05.910199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.910473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.910754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.912099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:05.914372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.060200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 984/1991] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:05 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:06.061281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.070433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b07ff4612e7420d84fe8d0b75e04c6b', 'request_id': 'req-eeff6cdb-2471-4ae5-929d-b764e6da0850'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.076858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b07ff4612e7420d84fe8d0b75e04c6b', 'request_id': 'req-eeff6cdb-2471-4ae5-929d-b764e6da0850'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.083412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b07ff4612e7420d84fe8d0b75e04c6b', 'request_id': 'req-eeff6cdb-2471-4ae5-929d-b764e6da0850'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.122551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371ef23-78d1-4d93-a404-aa1411f550b9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.127360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.128278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.128535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.128805 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.129436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:06.129757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.131320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.147911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 985/1992] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:06.147911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.147911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.171868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] The token cache is being invalidate because project 1b07ff4612e7420d84fe8d0b75e04c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:06.172243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.173037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.174641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1008/1993] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:05 2022] DELETE /identity/v3/projects/1b07ff4612e7420d84fe8d0b75e04c6b => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:06.175472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.176983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.209701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc9d8-6564-454c-9239-7330febf0399 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.213439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.214052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.214329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.214598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.216452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.228748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.230391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.239944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 986/1994] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:06.240742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.242052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.242626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6cdb-2471-4ae5-929d-b764e6da0850 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.247332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.248318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.248318 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.248512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.249262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.251272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.254342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8a66d-4338-492b-a6be-f28af207880b service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.260699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.261342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.261626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.261901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] PATH_INFO: `/v3/projects/ea6c284ac7324554996f7af605563fc7` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.263048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6c284ac7324554996f7af605563fc7)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.265034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.368671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1009/1995] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:06 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:06.369474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.371090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.383268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9874e3b-ca33-492e-91c0-3bde51b5c5ec admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.388720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6f8e1-919a-4f4c-a948-0236f2b630f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.389303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f8e1-919a-4f4c-a948-0236f2b630f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.389914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f8e1-919a-4f4c-a948-0236f2b630f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.392165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f8e1-919a-4f4c-a948-0236f2b630f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.394130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:06.394621 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6f8e1-919a-4f4c-a948-0236f2b630f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.396785 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6f8e1-919a-4f4c-a948-0236f2b630f9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.409808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6c284ac7324554996f7af605563fc7', 'request_id': 'req-ca392d63-1a04-4a60-a29d-2fbd21f89478'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.416140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6c284ac7324554996f7af605563fc7', 'request_id': 'req-ca392d63-1a04-4a60-a29d-2fbd21f89478'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.422546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6c284ac7324554996f7af605563fc7', 'request_id': 'req-ca392d63-1a04-4a60-a29d-2fbd21f89478'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.499272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] The token cache is being invalidate because project ea6c284ac7324554996f7af605563fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:06.499760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.502814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1010/1996] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:06 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:06.503343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.504656 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.505295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.505458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 987/1997] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:06 2022] DELETE /identity/v3/projects/ea6c284ac7324554996f7af605563fc7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:06.506080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.506533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.510581 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.533431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:06.535001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.589204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:06.608232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1011/1998] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:06.609313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.611142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.613635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca392d63-1a04-4a60-a29d-2fbd21f89478 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.618865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff3202-d425-45ae-9656-f6c36eeeed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.619609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff3202-d425-45ae-9656-f6c36eeeed30 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.619954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff3202-d425-45ae-9656-f6c36eeeed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.625075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff3202-d425-45ae-9656-f6c36eeeed30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.625075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff3202-d425-45ae-9656-f6c36eeeed30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.625075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff3202-d425-45ae-9656-f6c36eeeed30 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.625345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c4ef1-87d2-4a68-938c-ec78f4094c23 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.628969 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.629618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.629903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.630207 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.630836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.632778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.636228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 988/1999] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:06.636228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ca63-7073-4a47-a4e5-b0bbdc2b402b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.636228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ca63-7073-4a47-a4e5-b0bbdc2b402b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.636228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ca63-7073-4a47-a4e5-b0bbdc2b402b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.658287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8f31ca63-7073-4a47-a4e5-b0bbdc2b402b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:06.659691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8f31ca63-7073-4a47-a4e5-b0bbdc2b402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.668403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 989/2000] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:06.669625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393037b4-772a-4b42-9fa0-3877060d7cd8 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.670112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393037b4-772a-4b42-9fa0-3877060d7cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.670238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393037b4-772a-4b42-9fa0-3877060d7cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.689790 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-393037b4-772a-4b42-9fa0-3877060d7cd8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:06.690979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-393037b4-772a-4b42-9fa0-3877060d7cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.699683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 990/2001] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:06.700487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393037b4-772a-4b42-9fa0-3877060d7cd8 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.709792 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393037b4-772a-4b42-9fa0-3877060d7cd8 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.714959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6743f-c777-47b2-8230-74efa7b173ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.715565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6743f-c777-47b2-8230-74efa7b173ff admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.715877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6743f-c777-47b2-8230-74efa7b173ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.716156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6743f-c777-47b2-8230-74efa7b173ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.716794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6743f-c777-47b2-8230-74efa7b173ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.718418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6743f-c777-47b2-8230-74efa7b173ff admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.729388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 991/2002] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:06.730690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.730971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.731243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.751261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1012/2003] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:06 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:06.751261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.754455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:06.754455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.762015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b481e-689f-4878-a694-6c7d7e93fcfa admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.764311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 992/2004] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:06.764969 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.766293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:06.767252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.767771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.768151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.768431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.769075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:06.769387 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.771034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.778459 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7cc74-5d11-4560-836a-8e91b901f324 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.783317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.783809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1013/2005] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:06.784004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.784252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.784531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] PATH_INFO: `/v3/projects/0146872607c54eb48673395210465233` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.784664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.785613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0146872607c54eb48673395210465233)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.787523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.794454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdf6f8-b511-45b2-8b7c-fb0aec13e05c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:06.799036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:06.799685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:06.799992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:06.800383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] PATH_INFO: `/v3/projects/bc435b9609b545ca9d4d39304b7cb4f8/users/0060fa6e8bd349589558b79d84531431/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:06.816084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc435b9609b545ca9d4d39304b7cb4f8, user_id=0060fa6e8bd349589558b79d84531431, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:06.818764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.939056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1014/2006] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:06 2022] PUT /identity/v3/projects/bc435b9609b545ca9d4d39304b7cb4f8/users/0060fa6e8bd349589558b79d84531431/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:06.939895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:06.947456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0146872607c54eb48673395210465233', 'request_id': 'req-ae5a906c-6155-4185-994e-91af7110fd61'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.953124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0146872607c54eb48673395210465233', 'request_id': 'req-ae5a906c-6155-4185-994e-91af7110fd61'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:06.959419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0146872607c54eb48673395210465233', 'request_id': 'req-ae5a906c-6155-4185-994e-91af7110fd61'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:07.006817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74d39-0885-42ca-a5c8-b4031b4dd5c5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.053369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] The token cache is being invalidate because project 0146872607c54eb48673395210465233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:07.053665 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:07.054361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:07.056000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 993/2007] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:06 2022] DELETE /identity/v3/projects/0146872607c54eb48673395210465233 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:07.056721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.058055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.066152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.076594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.076594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.076594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.076594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.076594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.080780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1015/2008] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:07.081721 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.083381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.096496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba3bb8-f7f5-443d-b82a-c8f497b73a7c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.101118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.101714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.101993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.102263 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.102943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.105028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.126130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a906c-6155-4185-994e-91af7110fd61 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.131137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a9014-93c1-4d21-9553-ec4c97a91f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.131769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9014-93c1-4d21-9553-ec4c97a91f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.132079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9014-93c1-4d21-9553-ec4c97a91f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.132355 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9014-93c1-4d21-9553-ec4c97a91f09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.133004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a9014-93c1-4d21-9553-ec4c97a91f09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:07.133309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a9014-93c1-4d21-9553-ec4c97a91f09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.135466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a9014-93c1-4d21-9553-ec4c97a91f09 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.147107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 994/2009] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:07.148420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9296-c1c2-4a3c-b807-d6e51403fc23 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.148706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9296-c1c2-4a3c-b807-d6e51403fc23 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.148983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9296-c1c2-4a3c-b807-d6e51403fc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.177272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a38d9296-c1c2-4a3c-b807-d6e51403fc23 None None] MFA Rules not processed for user `0060fa6e8bd349589558b79d84531431`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:07.178538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a38d9296-c1c2-4a3c-b807-d6e51403fc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.243242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 995/2010] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:07.244464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.244753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.245030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.265598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:07.266891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.275561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 996/2011] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:07.276370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.277572 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.289143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b6e70-1bb4-498d-9906-37e057c59c6c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.294765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.295530 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.295841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.296134 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.296771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.298370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.304356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1016/2012] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:07.304356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.309527 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 997/2013] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:07.310136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.313909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.364907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77343764-5868-4b19-93c3-0314f5543a30 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.370504 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cbd90-749e-437d-a4a4-94c2724f6a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.371366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cbd90-749e-437d-a4a4-94c2724f6a0a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.371634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cbd90-749e-437d-a4a4-94c2724f6a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.371952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cbd90-749e-437d-a4a4-94c2724f6a0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.372052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e98fa-c010-4d37-9651-8b8e5a62fe0e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.372620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cbd90-749e-437d-a4a4-94c2724f6a0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.374561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cbd90-749e-437d-a4a4-94c2724f6a0a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.420364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.421003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.421285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.421579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.431537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:07.465676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1017/2014] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:07.467058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83affa31-ca1c-4ccb-99e8-d949eb004366 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.467340 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83affa31-ca1c-4ccb-99e8-d949eb004366 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.467669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83affa31-ca1c-4ccb-99e8-d949eb004366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.484911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.486696 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.488037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-83affa31-ca1c-4ccb-99e8-d949eb004366 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:07.489314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-83affa31-ca1c-4ccb-99e8-d949eb004366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.496743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 998/2015] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:07.497545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.512377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a066-d686-46a8-a165-c59073593bab service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.561835 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1018/2016] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:07.563008 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83affa31-ca1c-4ccb-99e8-d949eb004366 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.573644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83affa31-ca1c-4ccb-99e8-d949eb004366 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.578662 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.581425 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.582257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.582367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] PATH_INFO: `/v3/projects/c66a8e5b515f4ffa8434a5f203957f01/users/8d23e2dd025445da82f8e6bb9a51fafd/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.585155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.590206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.590206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.590206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.590206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.590206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.598829 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66a8e5b515f4ffa8434a5f203957f01, user_id=8d23e2dd025445da82f8e6bb9a51fafd, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.601832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.610552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:07.693329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 999/2017] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:07.693329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.699777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6da59-5f75-40e4-8ff6-e608d34984e8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.704459 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c239a-f10d-4624-8eaa-31455fb94f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.705042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c239a-f10d-4624-8eaa-31455fb94f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.705319 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c239a-f10d-4624-8eaa-31455fb94f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.705617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c239a-f10d-4624-8eaa-31455fb94f7a admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.706235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c239a-f10d-4624-8eaa-31455fb94f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.708243 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c239a-f10d-4624-8eaa-31455fb94f7a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.746062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1019/2018] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:07 2022] PUT /identity/v3/projects/c66a8e5b515f4ffa8434a5f203957f01/users/8d23e2dd025445da82f8e6bb9a51fafd/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:07.746968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.809821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f42e1-d8a3-4b8f-9480-682a1fe149b8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.814418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.815021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.815298 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.815566 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.816233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:07.816532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.818229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.830162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1000/2019] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:07.830162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51371bd-abe0-4171-adf5-d8c1ebc27350 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.830162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51371bd-abe0-4171-adf5-d8c1ebc27350 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.830162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51371bd-abe0-4171-adf5-d8c1ebc27350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.834353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1020/2020] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:07.834353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.844782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a778ad0-c56a-43e6-9b82-e42e1747c501 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.849459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2383cc-918b-46e2-bbd7-94374d7e15b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.850056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2383cc-918b-46e2-bbd7-94374d7e15b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.850334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2383cc-918b-46e2-bbd7-94374d7e15b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.850606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2383cc-918b-46e2-bbd7-94374d7e15b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.851237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2383cc-918b-46e2-bbd7-94374d7e15b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.852021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f51371bd-abe0-4171-adf5-d8c1ebc27350 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:07.852980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2383cc-918b-46e2-bbd7-94374d7e15b9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.853573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f51371bd-abe0-4171-adf5-d8c1ebc27350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.863757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1021/2021] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:07.865044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0902d3-ff4b-4fb6-8f94-d67b0fbeb2cc None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.865580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0902d3-ff4b-4fb6-8f94-d67b0fbeb2cc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.865827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0902d3-ff4b-4fb6-8f94-d67b0fbeb2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.894335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5a0902d3-ff4b-4fb6-8f94-d67b0fbeb2cc None None] MFA Rules not processed for user `8d23e2dd025445da82f8e6bb9a51fafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:07.895735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5a0902d3-ff4b-4fb6-8f94-d67b0fbeb2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:07.912638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1001/2022] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:07.920499 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51371bd-abe0-4171-adf5-d8c1ebc27350 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.925146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51371bd-abe0-4171-adf5-d8c1ebc27350 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.929844 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.930447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.930719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.931173 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.932443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.934061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.944309 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1002/2023] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:07.945128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.955819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec57ef-5bf7-4028-97fd-1f48d68789c7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.960387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.961010 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.961387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.961550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.962326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.964918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:07.969636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1022/2024] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:07.970625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0902d3-ff4b-4fb6-8f94-d67b0fbeb2cc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:07.982559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0902d3-ff4b-4fb6-8f94-d67b0fbeb2cc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:07.987487 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:07.988115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:07.988398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:07.988673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:07.989316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:07.991013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.001885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1023/2025] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:08.002774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.005751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:08.083633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1003/2026] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:07 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:08.083633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.083849 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206c53-c6d6-47d0-a5dd-1701bf2881a1 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.144812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.145432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.145742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.146011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.146909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79710fb-9834-4187-a131-b5b7f54a5d3d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.152618 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.153448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.153731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.154038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.154820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.156613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.167623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1004/2027] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:08.169087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.178911 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8fc60-d8aa-499a-99f9-097fafdc640c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.183775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.184393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.184676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.184951 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.185603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.187775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.208544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.210172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.219260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1024/2028] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:12:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:12:08.220073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.231065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629bfa2-5786-4609-a626-78af6de04935 service swift] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.236247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.240539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.240539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.240539 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] PATH_INFO: `/v3/projects/6ac2712fd0674a4a9104a6a778152ce7/users/58ed2b7d11a34fa386e22defcab690d8/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.252894 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac2712fd0674a4a9104a6a778152ce7, user_id=58ed2b7d11a34fa386e22defcab690d8, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.255828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.327889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1005/2029] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:08 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:08.327889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.391315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dd6f5-618d-495f-9a4c-2262e857b0d5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.394419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1025/2030] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:08 2022] PUT /identity/v3/projects/6ac2712fd0674a4a9104a6a778152ce7/users/58ed2b7d11a34fa386e22defcab690d8/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:08.395481 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.461078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d8f5-da5d-4424-984e-5d67bf4022a8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.466745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431bcae-be01-4b7b-a6e1-a305028827b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.467472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431bcae-be01-4b7b-a6e1-a305028827b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.467810 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431bcae-be01-4b7b-a6e1-a305028827b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.468014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431bcae-be01-4b7b-a6e1-a305028827b1 admin admin] PATH_INFO: `/v3/projects/3cea63489e004c349fd955adc32c4400/users/35beeaac0d01404597acefb86654bea2/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.476007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.477003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.477382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.477711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.478741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.480783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.483815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431bcae-be01-4b7b-a6e1-a305028827b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cea63489e004c349fd955adc32c4400, user_id=35beeaac0d01404597acefb86654bea2, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.486529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431bcae-be01-4b7b-a6e1-a305028827b1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.534372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:08.568381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1006/2031] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:08 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:08.569058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1026/2032] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:08 2022] PUT /identity/v3/projects/3cea63489e004c349fd955adc32c4400/users/35beeaac0d01404597acefb86654bea2/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:08.570465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741afaa5-363f-4ba0-a5e2-cfc85daa6493 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.570758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741afaa5-363f-4ba0-a5e2-cfc85daa6493 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.571051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741afaa5-363f-4ba0-a5e2-cfc85daa6493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.572613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.582943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b1034-df5e-432a-b1d2-7291494e4eb6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.609269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-741afaa5-363f-4ba0-a5e2-cfc85daa6493 None None] MFA Rules not processed for user `58ed2b7d11a34fa386e22defcab690d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:08.610855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-741afaa5-363f-4ba0-a5e2-cfc85daa6493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:08.646550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.647164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.647471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.647742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.648449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.651081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.683469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1027/2033] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:08.684404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741afaa5-363f-4ba0-a5e2-cfc85daa6493 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.695297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741afaa5-363f-4ba0-a5e2-cfc85daa6493 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.701200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e731212-4f08-47df-8989-5c7f00d81e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.701881 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e731212-4f08-47df-8989-5c7f00d81e47 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.702064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e731212-4f08-47df-8989-5c7f00d81e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.702330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e731212-4f08-47df-8989-5c7f00d81e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.702977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e731212-4f08-47df-8989-5c7f00d81e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.704687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e731212-4f08-47df-8989-5c7f00d81e47 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.723026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1028/2034] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:08.723026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add838b-ea44-4418-83a8-11bd4536c870 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.723026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add838b-ea44-4418-83a8-11bd4536c870 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.723026 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add838b-ea44-4418-83a8-11bd4536c870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.748443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1add838b-ea44-4418-83a8-11bd4536c870 None None] MFA Rules not processed for user `35beeaac0d01404597acefb86654bea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:08.749676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1add838b-ea44-4418-83a8-11bd4536c870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:08.778398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1007/2035] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:08 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:08.779216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.788830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdf689-46bd-40a7-b5df-8ded041e4a97 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.793824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.794425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.794705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.795053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.795727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.797796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.818856 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1029/2036] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:08.821005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add838b-ea44-4418-83a8-11bd4536c870 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.833626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add838b-ea44-4418-83a8-11bd4536c870 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.838216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.838839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.839120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.839394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.840095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.841732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.851472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1030/2037] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:08.852625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.862828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313cba-9097-45d8-a843-928c481a259e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.867593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.868180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.868476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.869281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.869907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:08.871482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:08.881541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1031/2038] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:08.882307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.883886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:08.921919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1008/2039] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:08 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:08.921919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.964436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a689e6-ae53-4195-957b-b220d932a21f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.987128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569d6b-fe52-4e6a-955e-d0942e12d6df admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:08.992120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:08.992813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.996535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.996535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] PATH_INFO: `/v3/projects/7beaa0a0543b4a09aee4231459641c06/users/c5fe2ed644fb4258a52b9c00e024831e/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.008754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7beaa0a0543b4a09aee4231459641c06, user_id=c5fe2ed644fb4258a52b9c00e024831e, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.012117 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.014065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9627e6a9e86e43b694f723bd4109460e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.014688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.014964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.015247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.075350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.075537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1009/2040] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:08 2022] PUT /identity/v3/projects/7beaa0a0543b4a09aee4231459641c06/users/c5fe2ed644fb4258a52b9c00e024831e/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:09.076214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.077295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.086916 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1032/2041] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:09.088998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.101622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c027210-8661-4336-9f5a-c11041e1720d service glance] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.156688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895f885-acfc-40c7-bab3-9cd4feb95bf9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.164255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70c4fe-1664-4d08-9a34-e7405d0cce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.165415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70c4fe-1664-4d08-9a34-e7405d0cce0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.165883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70c4fe-1664-4d08-9a34-e7405d0cce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.166318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70c4fe-1664-4d08-9a34-e7405d0cce0c admin admin] PATH_INFO: `/v3/projects/f4c6ff991e8d4c12bb4ee7d4c1860011/users/f0a6846df948460cb058bc02681c90b7/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.181749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.181870 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.182134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.182375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.183039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.185041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.185850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70c4fe-1664-4d08-9a34-e7405d0cce0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c6ff991e8d4c12bb4ee7d4c1860011, user_id=f0a6846df948460cb058bc02681c90b7, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.188948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70c4fe-1664-4d08-9a34-e7405d0cce0c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.338322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1010/2042] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:09 2022] PUT /identity/v3/projects/f4c6ff991e8d4c12bb4ee7d4c1860011/users/f0a6846df948460cb058bc02681c90b7/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:09.339733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41e1c2-08f4-4517-8d3f-875f8527a9cc None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.341279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41e1c2-08f4-4517-8d3f-875f8527a9cc None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.341279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41e1c2-08f4-4517-8d3f-875f8527a9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.363469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:09.368414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-df41e1c2-08f4-4517-8d3f-875f8527a9cc None None] MFA Rules not processed for user `c5fe2ed644fb4258a52b9c00e024831e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:09.371711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-df41e1c2-08f4-4517-8d3f-875f8527a9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:09.371843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1033/2043] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:09 2022] POST /identity/v3/users => generated 294 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:09.371843 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.383294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a63979-48e5-45be-a424-ba4c4114609c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.443468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1011/2044] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:09.447946 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41e1c2-08f4-4517-8d3f-875f8527a9cc None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.467412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.468071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.468354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.468800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.469500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.471117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.482303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1034/2045] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:09.484951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.492431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7fcbc-5f69-49b9-9332-1b6e46d90ee8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.497789 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd58035-330e-471f-8567-78bb83caf3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.498368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd58035-330e-471f-8567-78bb83caf3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.498641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd58035-330e-471f-8567-78bb83caf3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.498907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd58035-330e-471f-8567-78bb83caf3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.499516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd58035-330e-471f-8567-78bb83caf3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.501407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd58035-330e-471f-8567-78bb83caf3e0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.511641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41e1c2-08f4-4517-8d3f-875f8527a9cc None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.516456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.517073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.517352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.517629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.518270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.519945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.530737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1012/2046] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:09.531522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.541386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d322-a529-41e5-9e39-0ffc17bef118 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.545961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.546597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.546859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.547133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] PATH_INFO: `/v3/projects/f4c6ff991e8d4c12bb4ee7d4c1860011/users/f0a6846df948460cb058bc02681c90b7/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.549302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c6ff991e8d4c12bb4ee7d4c1860011, user_id=f0a6846df948460cb058bc02681c90b7, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.550687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:09.553677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.563993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1013/2047] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:09 2022] PUT /identity/v3/projects/f4c6ff991e8d4c12bb4ee7d4c1860011/users/f0a6846df948460cb058bc02681c90b7/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:09.564863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.625758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae8399-a0e6-458c-bf7d-5213f6b783c3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.630884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.631537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.631849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.632127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] PATH_INFO: `/v3/projects/9b2c34d3ec88442a9ac9b6b7c0b23960/users/a664d8013aa14b29a331b3d7facbd621/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.645640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b2c34d3ec88442a9ac9b6b7c0b23960, user_id=a664d8013aa14b29a331b3d7facbd621, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.648472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.656053 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1035/2048] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:09 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:09.657439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caca3a-3978-4354-9abc-56ecc40dace0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.657852 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caca3a-3978-4354-9abc-56ecc40dace0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.658013 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caca3a-3978-4354-9abc-56ecc40dace0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.691144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-86caca3a-3978-4354-9abc-56ecc40dace0 None None] MFA Rules not processed for user `f0a6846df948460cb058bc02681c90b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:09.691144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-86caca3a-3978-4354-9abc-56ecc40dace0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:09.718149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1014/2049] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:09 2022] PUT /identity/v3/projects/9b2c34d3ec88442a9ac9b6b7c0b23960/users/a664d8013aa14b29a331b3d7facbd621/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:09.722745 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.764809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1036/2050] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:09.766853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caca3a-3978-4354-9abc-56ecc40dace0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.789923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ad2a5-0b13-402b-9398-61758c930479 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.796508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.797126 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.797510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.797791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.798348 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.800424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.834170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caca3a-3978-4354-9abc-56ecc40dace0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.839538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.840463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.844024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.844024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.844024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.844024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.853860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1037/2051] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:09.857998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.867248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecec3ed-9595-4c75-a241-786c80848fa0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.871956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3ed60-35c9-4b1d-86ca-8efdf2f62fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.872591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3ed60-35c9-4b1d-86ca-8efdf2f62fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.872871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3ed60-35c9-4b1d-86ca-8efdf2f62fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.877238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3ed60-35c9-4b1d-86ca-8efdf2f62fe2 admin admin] PATH_INFO: `/v3/projects/9b2c34d3ec88442a9ac9b6b7c0b23960/users/a664d8013aa14b29a331b3d7facbd621/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.877238 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3ed60-35c9-4b1d-86ca-8efdf2f62fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b2c34d3ec88442a9ac9b6b7c0b23960, user_id=a664d8013aa14b29a331b3d7facbd621, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.877764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3ed60-35c9-4b1d-86ca-8efdf2f62fe2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.887781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1038/2052] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:09 2022] PUT /identity/v3/projects/9b2c34d3ec88442a9ac9b6b7c0b23960/users/a664d8013aa14b29a331b3d7facbd621/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:09.897683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.898184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.898704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.911819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1015/2053] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:09 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:09.914636 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:09.925437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ff930-1c5e-4c55-8649-34a7e60f9ad2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.930214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] MFA Rules not processed for user `a664d8013aa14b29a331b3d7facbd621`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:09.932980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:09.984883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.985626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.985992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.986334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.987200 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.989359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.000634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1016/2054] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:10.002272 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1039/2055] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:10.004887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.019599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7729d6c-5244-496d-acec-465c53e8c9ff admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.024587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.025824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.026155 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.026430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.027147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.028734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.040979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.042659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:10.051535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1017/2056] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:10.055648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.082257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb45d5-4b45-4ad5-844c-af97ce9d4ed7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.087405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19dfd-b9ca-4922-921a-1d8f150c7d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.088816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19dfd-b9ca-4922-921a-1d8f150c7d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.089102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19dfd-b9ca-4922-921a-1d8f150c7d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.089375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19dfd-b9ca-4922-921a-1d8f150c7d13 admin admin] PATH_INFO: `/v3/projects/904af38ad69a4933b37a7931ff583499/users/b12b58cec5d046d48dbffe8552cfe046/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.105031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19dfd-b9ca-4922-921a-1d8f150c7d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904af38ad69a4933b37a7931ff583499, user_id=b12b58cec5d046d48dbffe8552cfe046, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.107944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19dfd-b9ca-4922-921a-1d8f150c7d13 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.122416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1a91c-17a5-441c-a69e-6f778c8bd36d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.126157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.126788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.127216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.127493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.188734 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.190328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.201352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1040/2057] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:12:10.202205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.212041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d59d6-aae2-4cb3-8976-9968a440beef service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.226122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.226122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.226122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.226122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.268116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1018/2058] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:10 2022] PUT /identity/v3/projects/904af38ad69a4933b37a7931ff583499/users/b12b58cec5d046d48dbffe8552cfe046/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:10.290286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.290286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.299888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29bb6d-803c-46eb-8721-7a29a96417e2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.299888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29bb6d-803c-46eb-8721-7a29a96417e2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.299888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29bb6d-803c-46eb-8721-7a29a96417e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.304087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1041/2059] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:10.326017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ec29bb6d-803c-46eb-8721-7a29a96417e2 None None] MFA Rules not processed for user `b12b58cec5d046d48dbffe8552cfe046`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:10.338202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ec29bb6d-803c-46eb-8721-7a29a96417e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:10.419422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1019/2060] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:10.423711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.436901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04890c77-3a1b-4804-ae04-4a2c701e2a4d service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.498126 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de879e12-f661-4ad6-b4f9-ac58a125079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.498728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879e12-f661-4ad6-b4f9-ac58a125079a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.499006 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879e12-f661-4ad6-b4f9-ac58a125079a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.499279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879e12-f661-4ad6-b4f9-ac58a125079a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.499947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de879e12-f661-4ad6-b4f9-ac58a125079a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.501839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de879e12-f661-4ad6-b4f9-ac58a125079a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.549675 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:10.619645 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1042/2061] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:10 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:10.620164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29bb6d-803c-46eb-8721-7a29a96417e2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.631137 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29bb6d-803c-46eb-8721-7a29a96417e2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.710164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.710164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.710164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.710164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.715326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:10.717059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.723405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.839764 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412a0ad-d1d5-4fab-9a09-fb508c5e1034 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.840269 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412a0ad-d1d5-4fab-9a09-fb508c5e1034 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.840407 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412a0ad-d1d5-4fab-9a09-fb508c5e1034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.849316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1020/2062] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:10 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:10.853515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.860993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d412a0ad-d1d5-4fab-9a09-fb508c5e1034 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:10.862254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d412a0ad-d1d5-4fab-9a09-fb508c5e1034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:10.863596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ddad6-c90b-4f67-b53c-a5b85d86bd13 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.868775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.869469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.869802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.870308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.871081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.873705 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.885614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1021/2063] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:10.889638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:10.899552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8064f7-d7ab-4881-9910-7767f3708c93 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:10.905215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:10.905865 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:10.909426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:10.909426 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] PATH_INFO: `/v3/projects/1f8b4cc8ea064f528b759f51a3d60521/users/284202edecff4c03a80beffeab39cf6d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:10.919980 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1043/2064] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:12:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:10.921546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8b4cc8ea064f528b759f51a3d60521, user_id=284202edecff4c03a80beffeab39cf6d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:10.924308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:10.933168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412a0ad-d1d5-4fab-9a09-fb508c5e1034 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:11.002180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412a0ad-d1d5-4fab-9a09-fb508c5e1034 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:11.008357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:11.009034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.009922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.010198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.013420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1022/2065] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:10 2022] PUT /identity/v3/projects/1f8b4cc8ea064f528b759f51a3d60521/users/284202edecff4c03a80beffeab39cf6d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:11.017321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:11.019303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:11.021067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:11.030177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1044/2066] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:12:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 11:12:11.098931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48787248-1cb0-40a5-a269-308271affca8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:11.105949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc9656-91c0-4bd6-93e5-34db2a05271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:11.106635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc9656-91c0-4bd6-93e5-34db2a05271c admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.106919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc9656-91c0-4bd6-93e5-34db2a05271c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.107265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc9656-91c0-4bd6-93e5-34db2a05271c admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.107852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc9656-91c0-4bd6-93e5-34db2a05271c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:11.109616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc9656-91c0-4bd6-93e5-34db2a05271c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:11.121686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1023/2067] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:11.126442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:11.137862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe41b9e-76f2-42a5-97dc-fa5d2313e7a5 service swift] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:11.144389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:11.145001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.145286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.146417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] PATH_INFO: `/v3/projects/1f8b4cc8ea064f528b759f51a3d60521/users/284202edecff4c03a80beffeab39cf6d/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.149383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8b4cc8ea064f528b759f51a3d60521, user_id=284202edecff4c03a80beffeab39cf6d, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:11.154757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:11.169073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1045/2068] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:11 2022] PUT /identity/v3/projects/1f8b4cc8ea064f528b759f51a3d60521/users/284202edecff4c03a80beffeab39cf6d/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:11.180083 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd753-da0a-4198-80b1-79923903e1a0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.180322 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd753-da0a-4198-80b1-79923903e1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.180686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd753-da0a-4198-80b1-79923903e1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.214482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:11.224208 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d76cd753-da0a-4198-80b1-79923903e1a0 None None] MFA Rules not processed for user `284202edecff4c03a80beffeab39cf6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:11.225750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d76cd753-da0a-4198-80b1-79923903e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:11.282911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099eafda-0ca6-4f49-ad4e-f1daf1d63ded admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:11.287349 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:11.288005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.288284 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.288554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.321345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1024/2069] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:11.359205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:11.363896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:11.374297 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1046/2070] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:11.758151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:11.835750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cd753-da0a-4198-80b1-79923903e1a0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:11.847842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cd753-da0a-4198-80b1-79923903e1a0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:11.852193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:11.852908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.854020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.854678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.918415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:11.919891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:11.930921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1025/2071] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:11.932340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:12.687888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:12.689403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:12.788279 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c36b4-78c9-47fb-be14-009fda710b6b service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:12.794725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:12.795460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:12.795875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:12.796230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] PATH_INFO: `/v3/users/2be0803d7aa848ca991e56da69eb2e7a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:12.797500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be0803d7aa848ca991e56da69eb2e7a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:12.799511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:12.997567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:13.096851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be0803d7aa848ca991e56da69eb2e7a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.097530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be0803d7aa848ca991e56da69eb2e7a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.197183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be0803d7aa848ca991e56da69eb2e7a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.197807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be0803d7aa848ca991e56da69eb2e7a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.205575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4318b4ba-3d74-4a5a-a05b-2e8e8ec75cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be0803d7aa848ca991e56da69eb2e7a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.225049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1047/2072] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:12 2022] DELETE /identity/v3/users/2be0803d7aa848ca991e56da69eb2e7a => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:13.312489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:13.502599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:13.519852 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbd797-9f40-4c30-93f8-e3ef79a7e9c1 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:13.586377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:13.587006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:13.587281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:13.588714 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] PATH_INFO: `/v3/projects/d07b62a38d324811a951f04028cf2b77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:13.589888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07b62a38d324811a951f04028cf2b77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:13.591827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:13.690114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07b62a38d324811a951f04028cf2b77', 'request_id': 'req-67e77519-f524-4c83-909d-7d4f04ddf4a4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.695517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07b62a38d324811a951f04028cf2b77', 'request_id': 'req-67e77519-f524-4c83-909d-7d4f04ddf4a4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.702714 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07b62a38d324811a951f04028cf2b77', 'request_id': 'req-67e77519-f524-4c83-909d-7d4f04ddf4a4'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.808517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] The token cache is being invalidate because project d07b62a38d324811a951f04028cf2b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:13.808807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.809436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-67e77519-f524-4c83-909d-7d4f04ddf4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:13.810936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1026/2073] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:13 2022] DELETE /identity/v3/projects/d07b62a38d324811a951f04028cf2b77 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:13.815956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100848-7d7c-403d-9663-3bd0b9740749 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:13.816450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100848-7d7c-403d-9663-3bd0b9740749 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:13.816636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100848-7d7c-403d-9663-3bd0b9740749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:13.845111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-53100848-7d7c-403d-9663-3bd0b9740749 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:13.845836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-53100848-7d7c-403d-9663-3bd0b9740749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:13.906747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1048/2074] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:13.914180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e113e-589d-4c4e-bc89-a31c24a3770f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:13.914467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e113e-589d-4c4e-bc89-a31c24a3770f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:13.914745 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e113e-589d-4c4e-bc89-a31c24a3770f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:13.934266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-af8e113e-589d-4c4e-bc89-a31c24a3770f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:13.935470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-af8e113e-589d-4c4e-bc89-a31c24a3770f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:13.943716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1027/2075] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:13.948904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53100848-7d7c-403d-9663-3bd0b9740749 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:13.951548 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:13.958475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53100848-7d7c-403d-9663-3bd0b9740749 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:13.964277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:13.964875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:13.965154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:13.965426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:13.966068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:13.966388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:13.968007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:13.979647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1049/2076] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:13.983971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e113e-589d-4c4e-bc89-a31c24a3770f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:13.992929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e113e-589d-4c4e-bc89-a31c24a3770f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:13.998220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:13.998786 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:13.999057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:13.999326 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:13.999984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:14.001872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:14.113677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1028/2077] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:13 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:14.116914 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:14.128345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693a475-264c-4cde-a979-5192e59ef7a2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:14.185362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:14.185960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.186234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.186500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.187140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:14.189080 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:14.329133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1050/2078] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:14 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:14.333316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:14.343857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db080c5-b2bb-479e-952a-0e10785cdb14 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:14.349281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:14.349874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.350154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.350427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.351063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:14.352722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:14.362538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1029/2079] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:14.366452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:14.376906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566228-abc9-48c1-aa3a-bc07ab8a6751 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:14.381732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99396294-807f-4dc6-9694-320fb6703c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:14.382345 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99396294-807f-4dc6-9694-320fb6703c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.382620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99396294-807f-4dc6-9694-320fb6703c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.382890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99396294-807f-4dc6-9694-320fb6703c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.383527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99396294-807f-4dc6-9694-320fb6703c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:14.386030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99396294-807f-4dc6-9694-320fb6703c42 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:14.396398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1051/2080] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:14.400514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:14.410681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca155-3306-49d8-bf6c-71263425f907 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:14.415255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:14.416494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.416773 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.417163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] PATH_INFO: `/v3/projects/05e0a5b0576842e3993f35dea8c97d67/users/3238c1a1ea5149e98c1bc0739a350d0c/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.447291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e0a5b0576842e3993f35dea8c97d67, user_id=3238c1a1ea5149e98c1bc0739a350d0c, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:14.447291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:14.475502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:14.543811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1030/2081] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:14 2022] PUT /identity/v3/projects/05e0a5b0576842e3993f35dea8c97d67/users/3238c1a1ea5149e98c1bc0739a350d0c/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:14.554973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852a9a-0761-42de-9fa7-4c117ae6d26d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.555343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852a9a-0761-42de-9fa7-4c117ae6d26d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.555676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852a9a-0761-42de-9fa7-4c117ae6d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.585600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ab852a9a-0761-42de-9fa7-4c117ae6d26d None None] MFA Rules not processed for user `3238c1a1ea5149e98c1bc0739a350d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:14.586580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ab852a9a-0761-42de-9fa7-4c117ae6d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:14.651821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1052/2082] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:14.663516 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:14.665404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:14.725205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347fa1-4c6c-4af1-aeae-07e4adcc8339 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:14.728938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9627e6a9e86e43b694f723bd4109460e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:14.729537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.729812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.730082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.738918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:14.740537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:14.749782 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1031/2083] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:14.969068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:15.564038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:16.129941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:16.324129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:17.027065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:17.243125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:17.477589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:18.084684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:18.084684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:18.084684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:18.087333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:18.102957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:18.104840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:18.173084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1053/2084] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:12:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:18.192884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:18.194421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:18.269968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf595e3f-256a-4ccd-b47e-a4e3db1d0881 service glance] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:18.275168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:18.276393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:18.276668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:18.276956 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:18.285050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:18.286693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:18.295807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1032/2085] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:12:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:12:18.555935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:18.557855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:18.612866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee740612-19a5-403a-920f-3823fa0b43c0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:18.616593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:18.617202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:18.617478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:18.617752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:18.620361 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:18.688194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:18.689646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:18.701540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1054/2086] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:18.707211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:19.007051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:19.548685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:19.550164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:19.562214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3749dc-55d9-428d-b514-a951b06ee5ac service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:19.565791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:19.566385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:19.566661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:19.566931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:19.568624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:19.618111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:19.632971 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:19.634418 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:19.643649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1033/2087] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:20.192250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:20.615324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:20.875310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:21.146616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:21.544417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:21.667795 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:21.720601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:21.729763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccd361-b863-421f-aed3-f4f8a2501efe service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:21.733348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f96f5-ae10-4838-af21-f5831507ceea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:21.733944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f96f5-ae10-4838-af21-f5831507ceea service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:21.734218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f96f5-ae10-4838-af21-f5831507ceea service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:21.734490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f96f5-ae10-4838-af21-f5831507ceea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:21.736257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6d5f96f5-ae10-4838-af21-f5831507ceea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:21.800293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f96f5-ae10-4838-af21-f5831507ceea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:21.801728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f96f5-ae10-4838-af21-f5831507ceea service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:21.811924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1055/2088] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:22.163498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:22.297098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:22.300984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:22.355535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:22.362756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283dd7-1f26-497f-bc77-091b8cfd39c5 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:22.367305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:22.367964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:22.368245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:22.368518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] PATH_INFO: `/v3/users/a1a63a726a8349a1a382094b6410706c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:22.370144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a63a726a8349a1a382094b6410706c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:22.372091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:22.561072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a63a726a8349a1a382094b6410706c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:22.574350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a63a726a8349a1a382094b6410706c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:22.575066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a63a726a8349a1a382094b6410706c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:22.583352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a63a726a8349a1a382094b6410706c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:22.584390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a63a726a8349a1a382094b6410706c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:22.718009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1034/2089] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:22 2022] DELETE /identity/v3/users/a1a63a726a8349a1a382094b6410706c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:22.722520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:22.722846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:22.723117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:22.742850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:22.745076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:22.802975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1056/2090] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:22.813505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:22.814930 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:22.872202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae7580-f7ee-43b1-b5e3-205f243c3002 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:22.876199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986931e0-e058-4582-8e17-4efbaa1872fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:22.876753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986931e0-e058-4582-8e17-4efbaa1872fe service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:22.877165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986931e0-e058-4582-8e17-4efbaa1872fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:22.877321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986931e0-e058-4582-8e17-4efbaa1872fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:22.887217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986931e0-e058-4582-8e17-4efbaa1872fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:22.888983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986931e0-e058-4582-8e17-4efbaa1872fe service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:22.898927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1035/2091] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:22.918510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.026921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.180016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.328223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:23.329732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:23.342118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a970-cd54-498a-8f2f-75995a319010 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:23.346687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:23.347320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.347633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.347939 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] PATH_INFO: `/v3/projects/1fd929a4f1ec46ce98f73462f98ba453` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.349055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd929a4f1ec46ce98f73462f98ba453)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:23.351002 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:23.352559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.418812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.464800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd929a4f1ec46ce98f73462f98ba453', 'request_id': 'req-66900d8d-5346-4930-b3a2-8d1ac0756d10'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:23.470312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd929a4f1ec46ce98f73462f98ba453', 'request_id': 'req-66900d8d-5346-4930-b3a2-8d1ac0756d10'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:23.478314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd929a4f1ec46ce98f73462f98ba453', 'request_id': 'req-66900d8d-5346-4930-b3a2-8d1ac0756d10'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:23.550950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] The token cache is being invalidate because project 1fd929a4f1ec46ce98f73462f98ba453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:23.551233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:23.551880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-66900d8d-5346-4930-b3a2-8d1ac0756d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:23.556981 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1057/2092] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:23 2022] DELETE /identity/v3/projects/1fd929a4f1ec46ce98f73462f98ba453 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:23.558225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437742f-b569-4c74-9698-bdf6a59045e0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.558503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437742f-b569-4c74-9698-bdf6a59045e0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.558769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437742f-b569-4c74-9698-bdf6a59045e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.579294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b437742f-b569-4c74-9698-bdf6a59045e0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:23.580507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b437742f-b569-4c74-9698-bdf6a59045e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:23.612748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.643028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.643294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1036/2093] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:23.649848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc7589-c5ba-4cdb-aba8-7e69ce837de6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.651135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc7589-c5ba-4cdb-aba8-7e69ce837de6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.651249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc7589-c5ba-4cdb-aba8-7e69ce837de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.670678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bcbc7589-c5ba-4cdb-aba8-7e69ce837de6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:23.671824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bcbc7589-c5ba-4cdb-aba8-7e69ce837de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:23.680616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1058/2094] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:23.687549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437742f-b569-4c74-9698-bdf6a59045e0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:23.697882 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437742f-b569-4c74-9698-bdf6a59045e0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:23.703559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:23.704230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.704907 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.706236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.707107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:23.707636 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:23.709357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:23.722860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1037/2095] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:23.730240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc7589-c5ba-4cdb-aba8-7e69ce837de6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:23.734517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.739812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc7589-c5ba-4cdb-aba8-7e69ce837de6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:23.744781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:23.745390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.745664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.745929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.746550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:23.748562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:23.842299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1059/2096] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:23 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:23.846397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:23.856379 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27307bdf-cebd-47d5-92ef-3e3bbb2d0f7e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:23.862153 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.914874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:23.915480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.915766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.916122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.916802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:23.918737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:23.998224 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1038/2097] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:23 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:24.003047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:24.005063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:24.015853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64291ee-690d-4935-8665-8cddff3c06b6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:24.020543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:24.021138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:24.021419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:24.021718 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:24.022332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:24.023960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:24.034478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1060/2098] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:24.038834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:24.046981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:24.050580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec025f9c-2dc3-47f4-808b-4fc8f0ced01e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:24.055299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe167eea-2a68-4239-868c-c982d5da5777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:24.055926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe167eea-2a68-4239-868c-c982d5da5777 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:24.056209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe167eea-2a68-4239-868c-c982d5da5777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:24.057008 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe167eea-2a68-4239-868c-c982d5da5777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:24.057718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe167eea-2a68-4239-868c-c982d5da5777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:24.059397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe167eea-2a68-4239-868c-c982d5da5777 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:24.070642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1039/2099] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:24.077010 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:24.087413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29c0f5-a589-4b23-a019-54d76df10bd8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:24.092795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:24.093494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:24.093778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:24.094058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] PATH_INFO: `/v3/projects/46dee63e6ef54ae2a037d422a41dacf9/users/914b60bb9bfa471880700762562e24f1/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:24.109097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46dee63e6ef54ae2a037d422a41dacf9, user_id=914b60bb9bfa471880700762562e24f1, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:24.114257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:24.222055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1061/2100] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:24 2022] PUT /identity/v3/projects/46dee63e6ef54ae2a037d422a41dacf9/users/914b60bb9bfa471880700762562e24f1/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:24.227674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:24.227993 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:24.228266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:24.247517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:24.248889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:24.308052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1040/2101] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:24.317890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:24.319596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:24.329362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:24.377160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3e057-9a91-411d-9a5d-9a79e5bb0d5b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:24.381128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a60de5-c206-454d-b6ca-49d1bd1cf5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:24.381733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60de5-c206-454d-b6ca-49d1bd1cf5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:24.382011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60de5-c206-454d-b6ca-49d1bd1cf5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:24.382283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60de5-c206-454d-b6ca-49d1bd1cf5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:24.391953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a60de5-c206-454d-b6ca-49d1bd1cf5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:24.393377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a60de5-c206-454d-b6ca-49d1bd1cf5e6 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:24.403593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1062/2102] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:24.487253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:24.630781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:25.107077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:25.204241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:25.757059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:25.758587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:25.771758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc862ec0-9cc8-48eb-a7bf-ce10232e76f1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:25.777649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:25.778377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:25.778563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:25.778840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] PATH_INFO: `/v3/users/8d23e2dd025445da82f8e6bb9a51fafd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:25.779985 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d23e2dd025445da82f8e6bb9a51fafd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:25.782036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:25.809668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:25.923509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:25.924249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:25.924687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:25.962290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] MFA Rules not processed for user `914b60bb9bfa471880700762562e24f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:25.964074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:26.027392 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d23e2dd025445da82f8e6bb9a51fafd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:26.058964 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d23e2dd025445da82f8e6bb9a51fafd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:26.060302 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d23e2dd025445da82f8e6bb9a51fafd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:26.068441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d23e2dd025445da82f8e6bb9a51fafd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:26.069714 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-5a215269-adda-4690-84f8-4846ec716d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d23e2dd025445da82f8e6bb9a51fafd'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:26.087512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1063/2103] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:26.115382 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:26.117693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:26.129878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:26.177865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2abf07-c0af-40a6-9dbf-30d510a5c9ec None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:26.180834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1041/2104] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:25 2022] DELETE /identity/v3/users/8d23e2dd025445da82f8e6bb9a51fafd => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:26.182936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:26.183855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:26.184091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:26.184364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:26.185820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:26.186112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:26.186390 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:26.193449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:26.194966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:26.205779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1064/2105] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:12:26.207973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:26.209311 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:26.272218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1042/2106] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:26.288147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:26.289525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:26.302541 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4b594-fbf3-4699-a2ff-12506b65662a service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:26.356990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:26.357608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:26.357885 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:26.358154 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:26.366994 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:26.368495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:26.378195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1065/2107] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:26.893507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:26.901676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:26.903215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:26.938244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ca4e6-ca48-4221-a089-e55fda00e03a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:26.943616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:26.944260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:26.944726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:26.945009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] PATH_INFO: `/v3/projects/c66a8e5b515f4ffa8434a5f203957f01` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:26.947076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66a8e5b515f4ffa8434a5f203957f01)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:26.949094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:27.111413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66a8e5b515f4ffa8434a5f203957f01', 'request_id': 'req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.118469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66a8e5b515f4ffa8434a5f203957f01', 'request_id': 'req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.128464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66a8e5b515f4ffa8434a5f203957f01', 'request_id': 'req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.257503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] The token cache is being invalidate because project c66a8e5b515f4ffa8434a5f203957f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:27.257503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.257503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-7dbf7385-8cd6-42d2-b624-4178cc4e3ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.257503 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1043/2108] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:26 2022] DELETE /identity/v3/projects/c66a8e5b515f4ffa8434a5f203957f01 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:27.258241 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:27.268855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f7001-41c2-4ed2-ab5f-f95effbc556a service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:27.325919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:27.326545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:27.326836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:27.327105 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] PATH_INFO: `/v3/users/f0a6846df948460cb058bc02681c90b7` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:27.328247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a6846df948460cb058bc02681c90b7)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:27.330261 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:27.670646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a6846df948460cb058bc02681c90b7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.671326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a6846df948460cb058bc02681c90b7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.718408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:27.775620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a6846df948460cb058bc02681c90b7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.776609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a6846df948460cb058bc02681c90b7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.786658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a6846df948460cb058bc02681c90b7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:27.811422 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1066/2109] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:27 2022] DELETE /identity/v3/users/f0a6846df948460cb058bc02681c90b7 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:27.862735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:28.108413 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d32f-fd10-4ff0-890e-e8e387a5ff19 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:28.108951 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d32f-fd10-4ff0-890e-e8e387a5ff19 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:28.109122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d32f-fd10-4ff0-890e-e8e387a5ff19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:28.137556 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-7239d32f-fd10-4ff0-890e-e8e387a5ff19 None None] MFA Rules not processed for user `914b60bb9bfa471880700762562e24f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:28.139030 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7239d32f-fd10-4ff0-890e-e8e387a5ff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:28.209365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1044/2110] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:28.340070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:28.342814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:28.408437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883f16-2a7d-4dc1-99b3-b56a09889995 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:28.413177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:28.413809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:28.414091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:28.414366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] PATH_INFO: `/v3/projects/f4c6ff991e8d4c12bb4ee7d4c1860011` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:28.415473 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c6ff991e8d4c12bb4ee7d4c1860011)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:28.417519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:28.428723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:28.546745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c6ff991e8d4c12bb4ee7d4c1860011', 'request_id': 'req-fc18781f-8652-4dee-a202-513c60cfc10a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:28.552577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c6ff991e8d4c12bb4ee7d4c1860011', 'request_id': 'req-fc18781f-8652-4dee-a202-513c60cfc10a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:28.559636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c6ff991e8d4c12bb4ee7d4c1860011', 'request_id': 'req-fc18781f-8652-4dee-a202-513c60cfc10a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:28.570735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:28.667491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] The token cache is being invalidate because project f4c6ff991e8d4c12bb4ee7d4c1860011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:28.667491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:28.667759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc18781f-8652-4dee-a202-513c60cfc10a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:28.669271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1067/2111] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:28 2022] DELETE /identity/v3/projects/f4c6ff991e8d4c12bb4ee7d4c1860011 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:28.674278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc9c3b-ca67-44ab-a53f-7ee702590565 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:28.674565 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc9c3b-ca67-44ab-a53f-7ee702590565 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:28.674834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc9c3b-ca67-44ab-a53f-7ee702590565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:28.695175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-01bc9c3b-ca67-44ab-a53f-7ee702590565 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:28.696351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-01bc9c3b-ca67-44ab-a53f-7ee702590565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:28.756704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1045/2112] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:28.765095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d869a-dc4a-4dc3-821f-211582000905 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:28.766189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d869a-dc4a-4dc3-821f-211582000905 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:28.766516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d869a-dc4a-4dc3-821f-211582000905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:28.778352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:28.788383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ff1d869a-dc4a-4dc3-821f-211582000905 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:28.790204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ff1d869a-dc4a-4dc3-821f-211582000905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:28.798147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1068/2113] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:28.802584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc9c3b-ca67-44ab-a53f-7ee702590565 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:28.813748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc9c3b-ca67-44ab-a53f-7ee702590565 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:28.818898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:28.819526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:28.819832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:28.820119 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:28.820906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:28.821334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:28.823340 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:28.836750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1046/2114] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:28.841381 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d869a-dc4a-4dc3-821f-211582000905 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:28.850516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d869a-dc4a-4dc3-821f-211582000905 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:28.855963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:28.856581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:28.856860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:28.857133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:28.857761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:28.859682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:28.988072 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1069/2115] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:28 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:28.992330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.003045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d9bc2-0082-498b-93ba-53ce1d5bcc01 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.059121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.059732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.060042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.060315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.060968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.062928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.192919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1047/2116] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:28 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:29.201041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.214216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afef56-d309-48bd-a159-dbba4cc4f249 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.218475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.219179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.219455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.219721 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.220403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.221996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.236209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1070/2117] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:29.237376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.248067 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004200dd-109e-44b3-bf36-d44e976a0a58 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.252571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae3344-1395-4e14-af89-3c8779067ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.253164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae3344-1395-4e14-af89-3c8779067ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.253438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae3344-1395-4e14-af89-3c8779067ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.253706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae3344-1395-4e14-af89-3c8779067ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.254337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae3344-1395-4e14-af89-3c8779067ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.256215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae3344-1395-4e14-af89-3c8779067ce8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.267256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1048/2118] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:29.271455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.282412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2035e-8331-4516-af4f-7a4e5c409835 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.287034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.290661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.290661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.290661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] PATH_INFO: `/v3/projects/9ad9295bea0c4bcdae55010657218c01/users/8bdcb815630947bc996b9efbaa6a7eff/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.303283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9295bea0c4bcdae55010657218c01, user_id=8bdcb815630947bc996b9efbaa6a7eff, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.305972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.312460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:29.374911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1071/2119] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:29 2022] PUT /identity/v3/projects/9ad9295bea0c4bcdae55010657218c01/users/8bdcb815630947bc996b9efbaa6a7eff/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:29.391401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3eb4-561d-4440-8dca-90b36b8bccb5 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.391679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3eb4-561d-4440-8dca-90b36b8bccb5 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.391988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3eb4-561d-4440-8dca-90b36b8bccb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.420362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-8ddd3eb4-561d-4440-8dca-90b36b8bccb5 None None] MFA Rules not processed for user `8bdcb815630947bc996b9efbaa6a7eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:29.421562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-8ddd3eb4-561d-4440-8dca-90b36b8bccb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:29.484319 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1049/2120] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:29.488826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.552977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9378ba-d888-495c-b43a-4a3a373da8c7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.556116 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:29.559486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.560112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.560388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.560657 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.561277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.563350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.679544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1072/2121] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:29 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:29.684456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd3eb4-561d-4440-8dca-90b36b8bccb5 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.694982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd3eb4-561d-4440-8dca-90b36b8bccb5 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.750478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.750990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.751266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.751532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.752226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.754122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.769825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:29.871372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1050/2122] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:29 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:29.875429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.887265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f201ea-7d43-43a7-bd96-069797379866 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.891927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.892598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.892880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.893155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.893795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.895380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.906364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1073/2123] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:29.910514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.920574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7ea24-67ee-4c2b-b569-616b2b90d786 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.925165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.925801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.926080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.926352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] PATH_INFO: `/v3/projects/2dc6969d910a409783c8e217c17df3ff/users/779bc593a07e4b8f95aeaf4c2f2b8efb/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.941765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc6969d910a409783c8e217c17df3ff, user_id=779bc593a07e4b8f95aeaf4c2f2b8efb, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.944476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.969781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.026577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1051/2124] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:29 2022] PUT /identity/v3/projects/2dc6969d910a409783c8e217c17df3ff/users/779bc593a07e4b8f95aeaf4c2f2b8efb/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:30.030483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:30.040746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19894a-26f6-4ffe-827f-3027ef7d853a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:30.093027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.098997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47b978-456a-485b-9e83-bec4cbf7b882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:30.099598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47b978-456a-485b-9e83-bec4cbf7b882 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:30.099906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47b978-456a-485b-9e83-bec4cbf7b882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:30.100175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47b978-456a-485b-9e83-bec4cbf7b882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:30.100797 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47b978-456a-485b-9e83-bec4cbf7b882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:30.102375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47b978-456a-485b-9e83-bec4cbf7b882 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:30.112411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1074/2125] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:30.116591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:30.127124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a4e5-b89b-46ad-ae8b-352cd4bb5ff1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:30.132069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:30.132711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:30.132979 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:30.133338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] PATH_INFO: `/v3/projects/2dc6969d910a409783c8e217c17df3ff/users/779bc593a07e4b8f95aeaf4c2f2b8efb/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:30.135892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc6969d910a409783c8e217c17df3ff, user_id=779bc593a07e4b8f95aeaf4c2f2b8efb, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:30.136596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.142783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:30.149281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1052/2126] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:30 2022] PUT /identity/v3/projects/2dc6969d910a409783c8e217c17df3ff/users/779bc593a07e4b8f95aeaf4c2f2b8efb/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:30.159712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b36d1-7080-4916-a9db-fd6675dc5db4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:30.160024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b36d1-7080-4916-a9db-fd6675dc5db4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:30.160294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b36d1-7080-4916-a9db-fd6675dc5db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:30.189767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ad9b36d1-7080-4916-a9db-fd6675dc5db4 None None] MFA Rules not processed for user `779bc593a07e4b8f95aeaf4c2f2b8efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:30.190984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ad9b36d1-7080-4916-a9db-fd6675dc5db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:30.261698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1075/2127] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:30.272768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:30.274634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:30.332600 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca6cc7-ba3b-4fc8-85d4-972a4d3801f2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:30.336207 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45fafa-00eb-44a0-aba4-bde2270b2876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:30.336798 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45fafa-00eb-44a0-aba4-bde2270b2876 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:30.337071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45fafa-00eb-44a0-aba4-bde2270b2876 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:30.337339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45fafa-00eb-44a0-aba4-bde2270b2876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:30.346757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45fafa-00eb-44a0-aba4-bde2270b2876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:30.348281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45fafa-00eb-44a0-aba4-bde2270b2876 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:30.358122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1053/2128] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:30.415271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.455693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.505968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b36d1-7080-4916-a9db-fd6675dc5db4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:30.574391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b36d1-7080-4916-a9db-fd6675dc5db4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:30.579112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:30.580343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:30.580343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:30.580561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] PATH_INFO: `/v3/users/8bdcb815630947bc996b9efbaa6a7eff` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:30.581744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bdcb815630947bc996b9efbaa6a7eff)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:30.584001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:30.655308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.698663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.796027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bdcb815630947bc996b9efbaa6a7eff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:30.796686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bdcb815630947bc996b9efbaa6a7eff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:30.861024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bdcb815630947bc996b9efbaa6a7eff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:30.861676 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bdcb815630947bc996b9efbaa6a7eff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:30.871082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bdcb815630947bc996b9efbaa6a7eff'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:30.891981 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1076/2129] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:30 2022] DELETE /identity/v3/users/8bdcb815630947bc996b9efbaa6a7eff => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:30.896133 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:30.896417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:30.896686 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:30.918908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:30.920163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:30.970900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:30.982650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1054/2130] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:30.992622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:30.994133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:30.996588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:31.054073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80a30-bab2-4d25-8a28-347ddbfde54a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:31.057758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:31.058489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:31.058771 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:31.059046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:31.068088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:31.069510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:31.079492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1077/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:31.371605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:31.421119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:31.546271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:31.546271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:31.558541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a6dfe-f45d-472b-ae73-8c8a55d8572c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:31.563255 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:31.564381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:31.564838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:31.565120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] PATH_INFO: `/v3/projects/9ad9295bea0c4bcdae55010657218c01` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:31.566359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad9295bea0c4bcdae55010657218c01)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:31.568360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:31.721828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad9295bea0c4bcdae55010657218c01', 'request_id': 'req-3eebb47e-9853-42de-8f43-f04298bccfab'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:31.728402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad9295bea0c4bcdae55010657218c01', 'request_id': 'req-3eebb47e-9853-42de-8f43-f04298bccfab'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:31.735934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad9295bea0c4bcdae55010657218c01', 'request_id': 'req-3eebb47e-9853-42de-8f43-f04298bccfab'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:31.827038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] The token cache is being invalidate because project 9ad9295bea0c4bcdae55010657218c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:31.827331 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:31.827990 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:31.829550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1055/2132] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:31 2022] DELETE /identity/v3/projects/9ad9295bea0c4bcdae55010657218c01 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:31.833651 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:31.845066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57bd99-e6d2-480a-8b56-47b18878edb7 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:31.934820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:31.935472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:31.935753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:31.936203 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] PATH_INFO: `/v3/users/779bc593a07e4b8f95aeaf4c2f2b8efb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:31.937794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779bc593a07e4b8f95aeaf4c2f2b8efb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:31.940033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:32.178177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779bc593a07e4b8f95aeaf4c2f2b8efb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:32.178902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779bc593a07e4b8f95aeaf4c2f2b8efb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:32.298287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779bc593a07e4b8f95aeaf4c2f2b8efb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:32.298903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779bc593a07e4b8f95aeaf4c2f2b8efb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:32.306357 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b74f9659-d1e7-48eb-96a3-9f9d9f6df9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779bc593a07e4b8f95aeaf4c2f2b8efb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:32.326049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1078/2133] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:31 2022] DELETE /identity/v3/users/779bc593a07e4b8f95aeaf4c2f2b8efb => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:32.512570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:32.604612 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:32.737296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:32.738881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:32.752902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebb47e-9853-42de-8f43-f04298bccfab admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:32.811520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:32.812188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:32.812472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:32.812749 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] PATH_INFO: `/v3/projects/2dc6969d910a409783c8e217c17df3ff` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:32.813868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dc6969d910a409783c8e217c17df3ff)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:32.815807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:32.941522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dc6969d910a409783c8e217c17df3ff', 'request_id': 'req-21dc653a-4608-4b16-be37-923b56e2b1ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:32.947074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dc6969d910a409783c8e217c17df3ff', 'request_id': 'req-21dc653a-4608-4b16-be37-923b56e2b1ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:32.953785 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dc6969d910a409783c8e217c17df3ff', 'request_id': 'req-21dc653a-4608-4b16-be37-923b56e2b1ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:33.049534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] The token cache is being invalidate because project 2dc6969d910a409783c8e217c17df3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:33.049829 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:33.050496 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-21dc653a-4608-4b16-be37-923b56e2b1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:33.052337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1056/2134] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:32 2022] DELETE /identity/v3/projects/2dc6969d910a409783c8e217c17df3ff => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:33.057075 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a457e54-39b9-4b37-8d85-06f70e39cd76 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.057562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a457e54-39b9-4b37-8d85-06f70e39cd76 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.057665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a457e54-39b9-4b37-8d85-06f70e39cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.081160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5a457e54-39b9-4b37-8d85-06f70e39cd76 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:33.082418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5a457e54-39b9-4b37-8d85-06f70e39cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:33.082832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:33.117434 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:33.148996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1079/2135] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:33.158330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbd181-2ecd-4ed5-9ec2-79f9e345a7eb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.158616 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbd181-2ecd-4ed5-9ec2-79f9e345a7eb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.158895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbd181-2ecd-4ed5-9ec2-79f9e345a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.178973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-61dbd181-2ecd-4ed5-9ec2-79f9e345a7eb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:33.180197 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-61dbd181-2ecd-4ed5-9ec2-79f9e345a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:33.188128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1057/2136] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:33.192335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a457e54-39b9-4b37-8d85-06f70e39cd76 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:33.203077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a457e54-39b9-4b37-8d85-06f70e39cd76 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:33.207900 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:33.208530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.208809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.209082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.209727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:33.210031 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:33.211700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:33.224016 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1080/2137] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:33.228377 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbd181-2ecd-4ed5-9ec2-79f9e345a7eb None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:33.237427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbd181-2ecd-4ed5-9ec2-79f9e345a7eb None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:33.242095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:33.242683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.242962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.243230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.243880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:33.245799 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:33.353928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1058/2138] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:33 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:33.358125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:33.368128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbef798-e005-4b93-b011-58b745b35dab admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:33.428869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:33.429475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.429754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.430025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.430686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:33.432838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:33.443750 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:33.509897 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:33.552804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1081/2139] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:33 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:33.556922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:33.567699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54049c3-c8e0-499e-882c-0ca5ad73e079 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:33.572532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:33.573847 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.574215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.574491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.575127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:33.576765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:33.586747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1059/2140] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:33.590861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:33.600880 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8274f-d411-4232-a6df-cb82e28ab28c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:33.605741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c65c0f-1ba6-4ae5-85a6-f4ad1ef45dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:33.606277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c65c0f-1ba6-4ae5-85a6-f4ad1ef45dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.606550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c65c0f-1ba6-4ae5-85a6-f4ad1ef45dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.606816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c65c0f-1ba6-4ae5-85a6-f4ad1ef45dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.607445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c65c0f-1ba6-4ae5-85a6-f4ad1ef45dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:33.609130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c65c0f-1ba6-4ae5-85a6-f4ad1ef45dd6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:33.621156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1082/2141] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:33.625307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:33.636409 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c5ce6-f9ac-4c74-a587-95088afd0386 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:33.640712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:33.644436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.644436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.644436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] PATH_INFO: `/v3/projects/4d19026e49b448cca881b7de137dd934/users/18104836d3e747de87c9fca963f567d5/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.656643 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d19026e49b448cca881b7de137dd934, user_id=18104836d3e747de87c9fca963f567d5, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:33.659305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:33.745771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1060/2142] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:33 2022] PUT /identity/v3/projects/4d19026e49b448cca881b7de137dd934/users/18104836d3e747de87c9fca963f567d5/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:33.766169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.766453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.766746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.795275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] MFA Rules not processed for user `18104836d3e747de87c9fca963f567d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:33.796517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:33.860912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1083/2143] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:33.872804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:33.874504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:33.886932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:33.934639 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c9347-b156-49ca-bcbd-1e00aabf3322 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:33.938287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30223071-5922-4120-ba9f-6686f1791b00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:33.938872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223071-5922-4120-ba9f-6686f1791b00 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:33.939150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223071-5922-4120-ba9f-6686f1791b00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:33.939419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223071-5922-4120-ba9f-6686f1791b00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:33.947518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30223071-5922-4120-ba9f-6686f1791b00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:33.949286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30223071-5922-4120-ba9f-6686f1791b00 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:33.958318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1061/2144] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:34.021045 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:35.312826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:35.443516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:35.618061 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:35.619624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:35.699226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc9d27-50dc-4c9e-8157-b4346486749e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:35.703809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:35.704497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:35.705017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:35.705285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] PATH_INFO: `/v3/users/c5fe2ed644fb4258a52b9c00e024831e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:35.706485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5fe2ed644fb4258a52b9c00e024831e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:35.708592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:35.810003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:35.916822 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5fe2ed644fb4258a52b9c00e024831e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:35.917497 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5fe2ed644fb4258a52b9c00e024831e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:35.998219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:36.024277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5fe2ed644fb4258a52b9c00e024831e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.024987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5fe2ed644fb4258a52b9c00e024831e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.038167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5fe2ed644fb4258a52b9c00e024831e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.099100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1084/2145] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:35 2022] DELETE /identity/v3/users/c5fe2ed644fb4258a52b9c00e024831e => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:36.103664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:36.103664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:36.104070 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:36.124914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:36.126174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:36.187561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1062/2146] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:36.197749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:36.199940 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:36.258069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b89880-ea9c-470e-8a4c-e7bb813aac68 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:36.262246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:36.262840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:36.263115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:36.263385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:36.273040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:36.274527 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:36.284833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1085/2147] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:36.529867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:36.613588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:36.700905 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:36.702625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:36.714535 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05661f67-8d70-42d6-a62e-b0603e71348c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:36.719431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:36.720093 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:36.720366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:36.720640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] PATH_INFO: `/v3/projects/7beaa0a0543b4a09aee4231459641c06` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:36.721747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7beaa0a0543b4a09aee4231459641c06)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:36.723669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:36.845239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7beaa0a0543b4a09aee4231459641c06', 'request_id': 'req-02659ee9-b042-4131-a434-ff8c7c6bec76'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.850814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7beaa0a0543b4a09aee4231459641c06', 'request_id': 'req-02659ee9-b042-4131-a434-ff8c7c6bec76'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.857473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7beaa0a0543b4a09aee4231459641c06', 'request_id': 'req-02659ee9-b042-4131-a434-ff8c7c6bec76'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.977712 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] The token cache is being invalidate because project 7beaa0a0543b4a09aee4231459641c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:36.977929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.978543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:36.980625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1063/2148] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:36 2022] DELETE /identity/v3/projects/7beaa0a0543b4a09aee4231459641c06 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:36.984574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:36.994369 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c1748-7916-467d-a39a-16c093b81be9 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:37.051973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:37.052635 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:37.052915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:37.053181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] PATH_INFO: `/v3/users/b12b58cec5d046d48dbffe8552cfe046` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:37.054289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12b58cec5d046d48dbffe8552cfe046)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:37.056258 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:37.326050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12b58cec5d046d48dbffe8552cfe046'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:37.326625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12b58cec5d046d48dbffe8552cfe046'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:37.409364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12b58cec5d046d48dbffe8552cfe046'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:37.409981 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12b58cec5d046d48dbffe8552cfe046'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:37.418249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12b58cec5d046d48dbffe8552cfe046'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:37.435488 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1086/2149] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:36 2022] DELETE /identity/v3/users/b12b58cec5d046d48dbffe8552cfe046 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:37.499122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:37.906594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:37.908278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:37.922961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02659ee9-b042-4131-a434-ff8c7c6bec76 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:37.979799 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:37.980546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:37.980832 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:37.981108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] PATH_INFO: `/v3/projects/904af38ad69a4933b37a7931ff583499` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:37.982237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=904af38ad69a4933b37a7931ff583499)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:37.984275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:38.151046 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904af38ad69a4933b37a7931ff583499', 'request_id': 'req-3794b16e-81f5-4755-a23b-f90824bed0be'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.158428 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904af38ad69a4933b37a7931ff583499', 'request_id': 'req-3794b16e-81f5-4755-a23b-f90824bed0be'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.164827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904af38ad69a4933b37a7931ff583499', 'request_id': 'req-3794b16e-81f5-4755-a23b-f90824bed0be'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.210373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:38.265084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] The token cache is being invalidate because project 904af38ad69a4933b37a7931ff583499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:38.265374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.266009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.267536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1064/2150] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:37 2022] DELETE /identity/v3/projects/904af38ad69a4933b37a7931ff583499 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:38.271736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:38.273376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:38.286457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbe7dc-671f-4554-822d-16a46c89dde8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:38.342779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:38.343401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:38.343673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:38.343978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] PATH_INFO: `/v3/users/284202edecff4c03a80beffeab39cf6d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:38.345151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=284202edecff4c03a80beffeab39cf6d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:38.347120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:38.546898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:38.595766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284202edecff4c03a80beffeab39cf6d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.596546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284202edecff4c03a80beffeab39cf6d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.719935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284202edecff4c03a80beffeab39cf6d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.719935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284202edecff4c03a80beffeab39cf6d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.726468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4b2e97cf-5884-46d6-ba85-8ef4cd735a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284202edecff4c03a80beffeab39cf6d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:38.744323 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1087/2151] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:38 2022] DELETE /identity/v3/users/284202edecff4c03a80beffeab39cf6d => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:39.096068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:39.189744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:39.191321 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:39.205673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794b16e-81f5-4755-a23b-f90824bed0be admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:39.228893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:39.267276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:39.267940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:39.268216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:39.268486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] PATH_INFO: `/v3/projects/1f8b4cc8ea064f528b759f51a3d60521` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:39.269595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f8b4cc8ea064f528b759f51a3d60521)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:39.271585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:39.444753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f8b4cc8ea064f528b759f51a3d60521', 'request_id': 'req-212859c8-d423-479b-8b25-6047df6fb22b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:39.450779 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f8b4cc8ea064f528b759f51a3d60521', 'request_id': 'req-212859c8-d423-479b-8b25-6047df6fb22b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:39.457194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f8b4cc8ea064f528b759f51a3d60521', 'request_id': 'req-212859c8-d423-479b-8b25-6047df6fb22b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:39.530585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] The token cache is being invalidate because project 1f8b4cc8ea064f528b759f51a3d60521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:39.530883 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:39.531577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-212859c8-d423-479b-8b25-6047df6fb22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:39.533294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1065/2152] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:39 2022] DELETE /identity/v3/projects/1f8b4cc8ea064f528b759f51a3d60521 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:39.538119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934d4c5-2f70-4a99-8bdc-8ca93493691c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:39.538597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934d4c5-2f70-4a99-8bdc-8ca93493691c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:39.538790 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934d4c5-2f70-4a99-8bdc-8ca93493691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:39.553343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:39.559550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a934d4c5-2f70-4a99-8bdc-8ca93493691c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:39.560793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a934d4c5-2f70-4a99-8bdc-8ca93493691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:39.621615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1088/2153] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:39.630950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239cbf-426b-4ce3-872c-5733ba699375 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:39.631067 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239cbf-426b-4ce3-872c-5733ba699375 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:39.631277 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239cbf-426b-4ce3-872c-5733ba699375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:39.653549 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-73239cbf-426b-4ce3-872c-5733ba699375 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:39.654760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-73239cbf-426b-4ce3-872c-5733ba699375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:39.662707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1066/2154] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:39.667056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934d4c5-2f70-4a99-8bdc-8ca93493691c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:39.678022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934d4c5-2f70-4a99-8bdc-8ca93493691c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:39.682828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:39.683633 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:39.683953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:39.684223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:39.684874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:39.685171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:39.686834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:39.699646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1089/2155] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:12:39.704105 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239cbf-426b-4ce3-872c-5733ba699375 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:39.713349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239cbf-426b-4ce3-872c-5733ba699375 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:39.719435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:39.720069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:39.720345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:39.720613 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:39.721303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:39.723220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:39.724683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:39.824591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1067/2156] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:39 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:39.828286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:39.846593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a6937-65d4-4cff-ad7f-cfa11364d3c4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:39.896143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:39.919087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:39.919761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:39.920070 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:39.920367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:39.921005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:39.923663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:39.987521 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.051242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1090/2157] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:39 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:40.055351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.066892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43039899-ad6e-4311-855f-b569dd270c8a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.072779 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.073444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.075738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.075738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.075738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.077745 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:40.092360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1068/2158] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:40.093766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.104025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5f737-bd64-49cf-849d-4f23a595e191 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.109535 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b1d83-baed-45e3-af6e-49a0f69c30f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.110133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b1d83-baed-45e3-af6e-49a0f69c30f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.110437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b1d83-baed-45e3-af6e-49a0f69c30f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.110782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b1d83-baed-45e3-af6e-49a0f69c30f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.111499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b1d83-baed-45e3-af6e-49a0f69c30f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.113169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b1d83-baed-45e3-af6e-49a0f69c30f6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:40.124259 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1091/2159] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:40.128485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.139578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8115c4-5c91-4b3e-9f24-4d482ef360f2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.144560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.145132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.145488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.145771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] PATH_INFO: `/v3/projects/cc8d93270e4044079799e8cfa1882ac4/users/9f850368cfb0451fa12d2a2626562f5c/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.156344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.161850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8d93270e4044079799e8cfa1882ac4, user_id=9f850368cfb0451fa12d2a2626562f5c, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.164669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:40.190320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.279247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1069/2160] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:40 2022] PUT /identity/v3/projects/cc8d93270e4044079799e8cfa1882ac4/users/9f850368cfb0451fa12d2a2626562f5c/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:40.289598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a93dbd-e9ed-43b9-831e-cf793ff55988 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.289820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a93dbd-e9ed-43b9-831e-cf793ff55988 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.290089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a93dbd-e9ed-43b9-831e-cf793ff55988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.319708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-01a93dbd-e9ed-43b9-831e-cf793ff55988 None None] MFA Rules not processed for user `9f850368cfb0451fa12d2a2626562f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:40.320938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-01a93dbd-e9ed-43b9-831e-cf793ff55988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:40.386185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1092/2161] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:40.391174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.413683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.442407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.458952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2a21a-9f76-417b-b2c8-723b7bdcc968 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.463755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.464380 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.468272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.468596 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.469281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.471184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:40.577359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1070/2162] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:40 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:40.581614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a93dbd-e9ed-43b9-831e-cf793ff55988 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.593129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a93dbd-e9ed-43b9-831e-cf793ff55988 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.630737 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.651522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.652860 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.653196 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.653471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.654366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.656492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:40.661819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.787281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1093/2163] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:40 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:40.791514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.802473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86958b15-f611-4b46-891f-5272ba56114f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.806931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.807523 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.807828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.808222 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.809247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.810864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:40.820480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1071/2164] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:40.825510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.835312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de705dc-95ce-4b7a-b73b-f9ba6c8e9775 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.841077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b278028-dfe4-456f-9887-742f7a56f529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.841691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b278028-dfe4-456f-9887-742f7a56f529 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.841966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b278028-dfe4-456f-9887-742f7a56f529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.842233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b278028-dfe4-456f-9887-742f7a56f529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.842906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b278028-dfe4-456f-9887-742f7a56f529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.844519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b278028-dfe4-456f-9887-742f7a56f529 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:40.854863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1094/2165] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:40.856059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:40.858295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:40.868402 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6347e93-b428-41a4-95ac-43fefafa11e6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:40.873052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:40.873718 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:40.873997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:40.874266 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] PATH_INFO: `/v3/projects/5e1e88b54e41413a951037b176f9f4e0/users/dacc4a458cc94640ac93c999679f41ae/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:40.890168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1e88b54e41413a951037b176f9f4e0, user_id=dacc4a458cc94640ac93c999679f41ae, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:40.892900 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:41.009727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1072/2166] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:40 2022] PUT /identity/v3/projects/5e1e88b54e41413a951037b176f9f4e0/users/dacc4a458cc94640ac93c999679f41ae/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:41.020863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239579c-3eac-4c46-8cf6-e30689e8a9e2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.021152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239579c-3eac-4c46-8cf6-e30689e8a9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.021426 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239579c-3eac-4c46-8cf6-e30689e8a9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.050434 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c239579c-3eac-4c46-8cf6-e30689e8a9e2 None None] MFA Rules not processed for user `dacc4a458cc94640ac93c999679f41ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:41.051714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c239579c-3eac-4c46-8cf6-e30689e8a9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:41.116237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1095/2167] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:41.121449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:41.124183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:41.157066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:41.184009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653302ef-41ed-46f9-9708-25d3bbad3bb6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:41.188759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:41.189364 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.189642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.189915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.190375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:41.190550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:41.192508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:41.380537 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1073/2168] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:41 2022] POST /identity/v3/projects => generated 381 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:41.389338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239579c-3eac-4c46-8cf6-e30689e8a9e2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:41.396608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239579c-3eac-4c46-8cf6-e30689e8a9e2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:41.453700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:41.454309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.454592 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.454867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.455505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:41.457461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:41.554841 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1096/2169] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:41 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:41.556043 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:41.567757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95fdb8-f9a9-4523-b610-ef8b7df863fe admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:41.572248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:41.572841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.573116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.573385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.574564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:41.576252 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:41.586966 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1074/2170] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:41.591122 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:41.601109 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7571e50-2b9c-415b-ba16-f01419bf6662 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:41.606248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:41.606882 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.611572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.611572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] PATH_INFO: `/v3/projects/1899aceada474c73ac7df7ac03e5ca59/users/ad778be5afe64451b644f6a3a2557b86/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.626291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1899aceada474c73ac7df7ac03e5ca59, user_id=ad778be5afe64451b644f6a3a2557b86, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:41.629265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:41.661468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:41.729638 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:41.736445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1097/2171] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:41 2022] PUT /identity/v3/projects/1899aceada474c73ac7df7ac03e5ca59/users/ad778be5afe64451b644f6a3a2557b86/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:41.740361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:41.751415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190c573-8472-4e85-aae8-894d8a7a8c91 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:41.811141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202f1bc-4874-43ba-9f58-cbeafcc9039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:41.811943 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202f1bc-4874-43ba-9f58-cbeafcc9039b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.812592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202f1bc-4874-43ba-9f58-cbeafcc9039b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.812867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202f1bc-4874-43ba-9f58-cbeafcc9039b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.813611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202f1bc-4874-43ba-9f58-cbeafcc9039b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:41.815130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202f1bc-4874-43ba-9f58-cbeafcc9039b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:41.826346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1075/2172] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:41.830680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:41.841655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba85bf-4e7c-48ed-acb6-010afe1d603c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:41.846321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:41.846966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.847234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.847517 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] PATH_INFO: `/v3/projects/1899aceada474c73ac7df7ac03e5ca59/users/ad778be5afe64451b644f6a3a2557b86/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.849614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1899aceada474c73ac7df7ac03e5ca59, user_id=ad778be5afe64451b644f6a3a2557b86, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:41.852495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:41.864148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1098/2173] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:41 2022] PUT /identity/v3/projects/1899aceada474c73ac7df7ac03e5ca59/users/ad778be5afe64451b644f6a3a2557b86/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:41.935254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9595-d65e-464f-a3cf-0b30aaa9a59d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:41.935570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9595-d65e-464f-a3cf-0b30aaa9a59d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:41.935891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9595-d65e-464f-a3cf-0b30aaa9a59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:41.966121 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-57cf9595-d65e-464f-a3cf-0b30aaa9a59d None None] MFA Rules not processed for user `ad778be5afe64451b644f6a3a2557b86`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:41.966599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-57cf9595-d65e-464f-a3cf-0b30aaa9a59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:42.036113 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1076/2174] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:42.048076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:42.049850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:42.108094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdda38-6535-4439-8e40-5e4f543211d8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:42.112110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:42.112714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:42.113068 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:42.113249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:42.172718 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:42.174144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:42.183226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1099/2175] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:42.195826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:42.242466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:42.741997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:42.993428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:43.845846 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:43.944779 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:44.426465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf9595-d65e-464f-a3cf-0b30aaa9a59d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:44.436156 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf9595-d65e-464f-a3cf-0b30aaa9a59d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:44.439746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bb3b1-c9f8-4cd5-bea1-c01b11be60fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:44.440365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bb3b1-c9f8-4cd5-bea1-c01b11be60fe service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:44.440640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bb3b1-c9f8-4cd5-bea1-c01b11be60fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:44.440908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bb3b1-c9f8-4cd5-bea1-c01b11be60fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:44.499026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bb3b1-c9f8-4cd5-bea1-c01b11be60fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:44.500854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bb3b1-c9f8-4cd5-bea1-c01b11be60fe service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:44.518710 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1077/2176] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:44.627291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:44.821895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:45.339465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:45.341119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:45.410158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d049f-2aa8-4196-8890-6292b9301359 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:45.414670 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:45.415307 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:45.415584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:45.415888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] PATH_INFO: `/v3/users/18104836d3e747de87c9fca963f567d5` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:45.417011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] RBAC: Authorizing `identity:delete_user(user_id=18104836d3e747de87c9fca963f567d5)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:45.419593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:45.667118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:45.681168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18104836d3e747de87c9fca963f567d5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:45.681820 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18104836d3e747de87c9fca963f567d5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:45.764338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18104836d3e747de87c9fca963f567d5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:45.764965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18104836d3e747de87c9fca963f567d5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:45.772377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18104836d3e747de87c9fca963f567d5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:45.790175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1100/2177] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:45 2022] DELETE /identity/v3/users/18104836d3e747de87c9fca963f567d5 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:45.796149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:45.796431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:45.796702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:45.813181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:45.816495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:45.817752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:45.876653 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1078/2178] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:45.888724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:45.890174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:45.944987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:45.947101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b61ba-f41a-4dfb-896a-33c222a9388a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:45.951462 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cec4e-84f8-4529-9b6d-0ae50229588b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:45.952128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cec4e-84f8-4529-9b6d-0ae50229588b service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:45.952436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cec4e-84f8-4529-9b6d-0ae50229588b service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:45.952709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cec4e-84f8-4529-9b6d-0ae50229588b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:45.961745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cec4e-84f8-4529-9b6d-0ae50229588b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:45.963178 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cec4e-84f8-4529-9b6d-0ae50229588b service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:45.973394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1101/2179] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:46.284645 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:46.514628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:46.520417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:46.536076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1ecb4-80e4-462d-ad44-e5aa68c64882 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:46.549531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:46.550199 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:46.550478 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:46.550748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] PATH_INFO: `/v3/projects/4d19026e49b448cca881b7de137dd934` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:46.551967 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d19026e49b448cca881b7de137dd934)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:46.553885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:46.688062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:46.708176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d19026e49b448cca881b7de137dd934', 'request_id': 'req-08d32117-6590-457a-8bc4-cb73d2d39205'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:46.714018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d19026e49b448cca881b7de137dd934', 'request_id': 'req-08d32117-6590-457a-8bc4-cb73d2d39205'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:46.720922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d19026e49b448cca881b7de137dd934', 'request_id': 'req-08d32117-6590-457a-8bc4-cb73d2d39205'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:46.814301 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] The token cache is being invalidate because project 4d19026e49b448cca881b7de137dd934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:46.814588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:46.815240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-08d32117-6590-457a-8bc4-cb73d2d39205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:46.816821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1079/2180] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:46 2022] DELETE /identity/v3/projects/4d19026e49b448cca881b7de137dd934 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:46.821735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa2dc-0cf4-454d-8176-feb0f84a3cae None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:46.822226 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa2dc-0cf4-454d-8176-feb0f84a3cae None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:46.822323 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa2dc-0cf4-454d-8176-feb0f84a3cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:46.847983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-736fa2dc-0cf4-454d-8176-feb0f84a3cae None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:46.849715 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-736fa2dc-0cf4-454d-8176-feb0f84a3cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:46.914839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1102/2181] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:46.923227 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd41ff-3db0-4dad-b632-bb0abe97fdfe None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:46.923597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd41ff-3db0-4dad-b632-bb0abe97fdfe None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:46.923895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd41ff-3db0-4dad-b632-bb0abe97fdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:46.946333 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-02bd41ff-3db0-4dad-b632-bb0abe97fdfe None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:46.947569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-02bd41ff-3db0-4dad-b632-bb0abe97fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:46.955672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1080/2182] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:46.960088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fa2dc-0cf4-454d-8176-feb0f84a3cae None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:46.969615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fa2dc-0cf4-454d-8176-feb0f84a3cae None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:46.974320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:46.974942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:46.975221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:46.975494 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:46.976186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:46.976485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:46.978076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:46.990351 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1103/2183] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:46.994789 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd41ff-3db0-4dad-b632-bb0abe97fdfe None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.005171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd41ff-3db0-4dad-b632-bb0abe97fdfe None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.010212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.010827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.011108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.011382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.012104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.014062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.040691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.042520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.053009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:47.058997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eec5e9-5d1d-4e3e-88db-ebed185981f5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.076476 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.077217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.077411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.077680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] PATH_INFO: `/v3/users/abc2334096894d3ca9c7433e4d8f9cd2` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.078845 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc2334096894d3ca9c7433e4d8f9cd2)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.080965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.120589 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1081/2184] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:46 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:47.124507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.188555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e73de-fb17-4767-a17a-e34eecf3c3a4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.193894 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.198071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.198071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.198071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.198071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.198071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.350289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1082/2185] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:47 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:47.351160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.353047 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.390953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc2334096894d3ca9c7433e4d8f9cd2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:47.391674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc2334096894d3ca9c7433e4d8f9cd2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:47.414054 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056aa80-e94e-4689-a9cd-4d084307ad60 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.417731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.418415 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.418692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.418961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.423074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.483317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.484868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.494583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1083/2186] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:47.495369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.506246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fe469-ddea-4288-a61d-a48413f0094d service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.511245 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.514100 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.514443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.514730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.515371 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.517036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.519569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc2334096894d3ca9c7433e4d8f9cd2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:47.520245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc2334096894d3ca9c7433e4d8f9cd2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:47.530796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1084/2187] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:47.531850 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.532232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fec7ee2c-46f4-4018-bf95-1c4db2d071bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc2334096894d3ca9c7433e4d8f9cd2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:47.533540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.546975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fabe51-ba89-4bb8-8711-b99b744dd249 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.550291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1104/2188] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:47 2022] DELETE /identity/v3/users/abc2334096894d3ca9c7433e4d8f9cd2 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:47.554647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.554786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.555000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.555361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.555478 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.555628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.555924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.556666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.558276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.573028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1085/2189] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:47.576949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:47.577788 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.578133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.587225 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d342-8bcc-43f2-bbc3-c27b51b40706 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.610011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:47.642086 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1105/2190] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:47.648187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7aee1-3001-4030-865f-6a05ab0c161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.648881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7aee1-3001-4030-865f-6a05ab0c161a admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.649240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7aee1-3001-4030-865f-6a05ab0c161a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.649515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7aee1-3001-4030-865f-6a05ab0c161a admin admin] PATH_INFO: `/v3/projects/1f14e4ba399d4349a0fe9ee6f7b894ed/users/ce7478c14d9245b9bc75a63af96231d9/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.653783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.655276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.665157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7aee1-3001-4030-865f-6a05ab0c161a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f14e4ba399d4349a0fe9ee6f7b894ed, user_id=ce7478c14d9245b9bc75a63af96231d9, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.667869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7aee1-3001-4030-865f-6a05ab0c161a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.724091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bd40b-5b92-4d02-92f2-fcd0719c97af None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.727916 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.728640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.729024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.729408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.739561 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:47.741256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:47.746432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1086/2191] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:47 2022] PUT /identity/v3/projects/1f14e4ba399d4349a0fe9ee6f7b894ed/users/ce7478c14d9245b9bc75a63af96231d9/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:47.759525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1106/2192] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:47.763006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.763296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.763578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.794462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] MFA Rules not processed for user `ce7478c14d9245b9bc75a63af96231d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:47.795849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.863545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1087/2193] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:47.875136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:47.876605 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:47.935169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b664cd-7932-4167-8ae6-3cf2893fceff service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:47.987722 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:47.988378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:47.988717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:47.989012 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:47.991385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:48.000246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:48.001818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:48.011334 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1107/2194] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:48.323770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:48.325802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:48.351353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:48.408991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60873f-b73f-409a-99fe-7708ade6b41c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:48.413652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:48.414279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:48.414559 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:48.414834 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] PATH_INFO: `/v3/projects/ded503bdd34b498da05cc322ac7f84a0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:48.416048 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded503bdd34b498da05cc322ac7f84a0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:48.418532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:48.579816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded503bdd34b498da05cc322ac7f84a0', 'request_id': 'req-94cf3556-6ba7-483f-b15a-8bde2ab09c20'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:48.586096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded503bdd34b498da05cc322ac7f84a0', 'request_id': 'req-94cf3556-6ba7-483f-b15a-8bde2ab09c20'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:48.593149 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded503bdd34b498da05cc322ac7f84a0', 'request_id': 'req-94cf3556-6ba7-483f-b15a-8bde2ab09c20'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:48.724009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] The token cache is being invalidate because project ded503bdd34b498da05cc322ac7f84a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:48.724009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:48.724009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:48.724009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1088/2195] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:48 2022] DELETE /identity/v3/projects/ded503bdd34b498da05cc322ac7f84a0 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:48.724377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:48.734405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a0997-14a8-4dcb-bd8d-5ad8e7754f12 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:48.744124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:48.828197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:48.828846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:48.829143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:48.829440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] PATH_INFO: `/v3/users/b9dcfaa0a5e041a286cbdd0bb9745444` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:48.830600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9dcfaa0a5e041a286cbdd0bb9745444)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:48.832681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:49.093736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9dcfaa0a5e041a286cbdd0bb9745444'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:49.094454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9dcfaa0a5e041a286cbdd0bb9745444'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:49.213058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9dcfaa0a5e041a286cbdd0bb9745444'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:49.213685 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9dcfaa0a5e041a286cbdd0bb9745444'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:49.222164 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33a2c384-54d2-466c-9084-037bce18b0b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9dcfaa0a5e041a286cbdd0bb9745444'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:49.242014 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1108/2196] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:48 2022] DELETE /identity/v3/users/b9dcfaa0a5e041a286cbdd0bb9745444 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:49.322367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:49.441674 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:49.800588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:49.802151 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:49.816534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3556-6ba7-483f-b15a-8bde2ab09c20 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:49.888098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:49.888764 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:49.889067 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:49.889336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] PATH_INFO: `/v3/projects/0286c81abf874ed49fbc13c010241a13` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:49.890507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0286c81abf874ed49fbc13c010241a13)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:49.892463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:49.922338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:50.110442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:50.154470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0286c81abf874ed49fbc13c010241a13', 'request_id': 'req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:50.160195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0286c81abf874ed49fbc13c010241a13', 'request_id': 'req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:50.166654 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0286c81abf874ed49fbc13c010241a13', 'request_id': 'req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:50.301983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] The token cache is being invalidate because project 0286c81abf874ed49fbc13c010241a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:50.302276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:50.302903 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6a92f47a-f6ac-4adc-97e5-49b057d47a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:50.304853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1089/2197] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:49 2022] DELETE /identity/v3/projects/0286c81abf874ed49fbc13c010241a13 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:50.311177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab034-6f2b-48c1-a464-f18701419b18 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.311428 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab034-6f2b-48c1-a464-f18701419b18 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.311728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab034-6f2b-48c1-a464-f18701419b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.332730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d60ab034-6f2b-48c1-a464-f18701419b18 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:50.334011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d60ab034-6f2b-48c1-a464-f18701419b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:50.393356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1109/2198] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:50.401422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4418-173d-49ac-b7ba-a674fe21557f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.401961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4418-173d-49ac-b7ba-a674fe21557f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.401961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4418-173d-49ac-b7ba-a674fe21557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.425660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2d5e4418-173d-49ac-b7ba-a674fe21557f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:50.426879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2d5e4418-173d-49ac-b7ba-a674fe21557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:50.435701 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1090/2199] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:50.440129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab034-6f2b-48c1-a464-f18701419b18 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.451362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab034-6f2b-48c1-a464-f18701419b18 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.456158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.456807 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.457084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.457354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.458188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:50.458677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.460342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:50.476686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1110/2200] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:50.481819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4418-173d-49ac-b7ba-a674fe21557f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.490857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4418-173d-49ac-b7ba-a674fe21557f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.496876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.497013 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.497308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.497457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.498081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.500022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:50.557020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:50.566801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.569143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:50.598752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95b77-bbea-4a65-b308-0a34a065dfa3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.605225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.607289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.607964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.608503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] PATH_INFO: `/v3/users/0060fa6e8bd349589558b79d84531431` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.610919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0060fa6e8bd349589558b79d84531431)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.613588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:50.630638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1091/2201] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:50 2022] POST /identity/v3/projects => generated 407 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:50.634560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.714193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ceb8c-3499-4b14-a875-1a7055ec80fb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.719036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.719647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.720050 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.720307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.720940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.722864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:50.850152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1092/2202] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:50 2022] POST /identity/v3/users => generated 304 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:50.854192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.864730 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a5a51-d2bf-4db0-987c-b0c98b7f5001 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.869373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.869958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.870232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.870502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.871284 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.872869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:50.882944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1093/2203] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:50.890904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.900909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54035db7-fbc7-44c0-ad98-a05642b4f65b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.907619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.908349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.908656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.908901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.909552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.911270 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:50.925263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1094/2204] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:12:50.929515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.940259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af792-ff84-4403-ae33-276008c63ad4 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.944851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485db79-85b1-4e09-81d8-a3655841bcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.945517 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485db79-85b1-4e09-81d8-a3655841bcec admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.945932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485db79-85b1-4e09-81d8-a3655841bcec admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.946211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485db79-85b1-4e09-81d8-a3655841bcec admin admin] PATH_INFO: `/v3/projects/404e1b1ea8974c95a038f43fda656503/users/237af7a747604f83b59e3aa8c4a240f1/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.961143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485db79-85b1-4e09-81d8-a3655841bcec admin admin] RBAC: Authorizing `identity:create_grant(project_id=404e1b1ea8974c95a038f43fda656503, user_id=237af7a747604f83b59e3aa8c4a240f1, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.961323 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0060fa6e8bd349589558b79d84531431'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:50.961788 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0060fa6e8bd349589558b79d84531431'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:50.963958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485db79-85b1-4e09-81d8-a3655841bcec admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.078949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0060fa6e8bd349589558b79d84531431'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:51.079663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0060fa6e8bd349589558b79d84531431'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:51.085676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1095/2205] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:50 2022] PUT /identity/v3/projects/404e1b1ea8974c95a038f43fda656503/users/237af7a747604f83b59e3aa8c4a240f1/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:51.087655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ea1ca095-e6e4-46ef-b15b-713f8f5ab916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0060fa6e8bd349589558b79d84531431'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:51.098267 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.098655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.098936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.109182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1111/2206] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:50 2022] DELETE /identity/v3/users/0060fa6e8bd349589558b79d84531431 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:51.114949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.115538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.115538 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.127919 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] MFA Rules not processed for user `237af7a747604f83b59e3aa8c4a240f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:51.129152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:51.134737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:51.135938 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:51.170266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:51.198325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1096/2207] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:51.205707 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1112/2208] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:51.216866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.216866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:51.219452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.220954 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:51.224961 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb39e1a-e386-4b47-8c21-7ca6e3ac9a66 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.232122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.233160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.233505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.233959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.234826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.237925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.280060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef17e2-c7b8-4f7b-a1c9-51561aac6f03 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.283608 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.284239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.284518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.284787 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.294641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.296101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.305912 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1113/2209] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:51.418925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1097/2210] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:51 2022] POST /identity/v3/projects => generated 407 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:51.431446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.442060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d02a9-eb55-4164-bee2-da3b909d435e service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.498311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.498922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.500500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.500500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.500500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.502219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.621696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1114/2211] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:51 2022] POST /identity/v3/users => generated 310 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:51.624545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.635286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e183fa-1b23-448c-8363-4f818ee3f382 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.641026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.641886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.642103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.642358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.643004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.644738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.662178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1098/2212] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:51.669189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.690595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174992c-47aa-425d-98d5-a301134b5800 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.695738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.696504 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.696786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.697055 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] PATH_INFO: `/v3/projects/4814a58897e342a4b37b7c121e191f26/users/4e9c5df2f8314b7abc2291cea86bdb5f/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.722996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4814a58897e342a4b37b7c121e191f26, user_id=4e9c5df2f8314b7abc2291cea86bdb5f, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.725795 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.740826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.742888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:51.763363 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad574b3-5bfe-4e94-a825-c9c171951aed admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.771684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.772487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.772863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.773176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] PATH_INFO: `/v3/projects/bc435b9609b545ca9d4d39304b7cb4f8` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.774551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc435b9609b545ca9d4d39304b7cb4f8)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.776806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.796739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1115/2213] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:51 2022] PUT /identity/v3/projects/4814a58897e342a4b37b7c121e191f26/users/4e9c5df2f8314b7abc2291cea86bdb5f/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:51.800972 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.867368 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edb3b6-92b1-4df6-ab12-3ee4c2a8b299 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.872842 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.873450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.873732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.874004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.874777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.876413 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.892071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1116/2214] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:51.894698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:51.905166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed5807-599f-4290-b24f-562ed9027c7e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:51.908595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc435b9609b545ca9d4d39304b7cb4f8', 'request_id': 'req-832a8055-79de-44f1-a889-0c9410644361'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:51.912801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:51.917480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc435b9609b545ca9d4d39304b7cb4f8', 'request_id': 'req-832a8055-79de-44f1-a889-0c9410644361'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:51.925351 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc435b9609b545ca9d4d39304b7cb4f8', 'request_id': 'req-832a8055-79de-44f1-a889-0c9410644361'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:51.966379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996e0f3-4b74-4f50-ad10-f1ba52a8c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:51.966966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996e0f3-4b74-4f50-ad10-f1ba52a8c7c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.967248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996e0f3-4b74-4f50-ad10-f1ba52a8c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.967529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996e0f3-4b74-4f50-ad10-f1ba52a8c7c3 admin admin] PATH_INFO: `/v3/projects/4814a58897e342a4b37b7c121e191f26/users/4e9c5df2f8314b7abc2291cea86bdb5f/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:51.969556 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996e0f3-4b74-4f50-ad10-f1ba52a8c7c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4814a58897e342a4b37b7c121e191f26, user_id=4e9c5df2f8314b7abc2291cea86bdb5f, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:51.972404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996e0f3-4b74-4f50-ad10-f1ba52a8c7c3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:51.983106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1117/2215] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:51 2022] PUT /identity/v3/projects/4814a58897e342a4b37b7c121e191f26/users/4e9c5df2f8314b7abc2291cea86bdb5f/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:51.993152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:51.993544 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:51.993824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.004014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] The token cache is being invalidate because project bc435b9609b545ca9d4d39304b7cb4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:52.009977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:52.009977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-832a8055-79de-44f1-a889-0c9410644361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:52.009977 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1099/2216] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:51 2022] DELETE /identity/v3/projects/bc435b9609b545ca9d4d39304b7cb4f8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:52.014299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daff786-6fda-4c1e-9df3-1b73d4b6eb5d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.014584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daff786-6fda-4c1e-9df3-1b73d4b6eb5d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.014859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daff786-6fda-4c1e-9df3-1b73d4b6eb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.023805 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] MFA Rules not processed for user `4e9c5df2f8314b7abc2291cea86bdb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:52.025038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:52.038650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-2daff786-6fda-4c1e-9df3-1b73d4b6eb5d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:52.038650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2daff786-6fda-4c1e-9df3-1b73d4b6eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:52.109715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1100/2217] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:52.115380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1118/2218] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:52.119265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9038d-107a-4978-bd62-bd3ebb707401 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.119265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9038d-107a-4978-bd62-bd3ebb707401 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.119265 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9038d-107a-4978-bd62-bd3ebb707401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.126552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.127979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:52.140106 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-55c9038d-107a-4978-bd62-bd3ebb707401 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:52.141312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-55c9038d-107a-4978-bd62-bd3ebb707401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:52.150148 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1101/2219] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:52.154796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9038d-107a-4978-bd62-bd3ebb707401 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.164304 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9038d-107a-4978-bd62-bd3ebb707401 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.170274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.170860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.171136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.171986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.172159 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:52.172462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.174094 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.186027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1102/2220] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:12:52.187034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d2e93-d561-4e50-b3e3-7741a59c26b5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.191064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.191448 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.192090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.192386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.192648 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.201453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e63dac-4e0e-4023-9e91-536de4092c20 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.202257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.203701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.206722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.207313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.207592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.207895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.208524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.210429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.213208 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1119/2221] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:52 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:12:52.259460 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:52.383312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1103/2222] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:52 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:52.388199 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.399817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e7e60-8c5a-4c32-a3ad-2f0cdddc8b96 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.457411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.458007 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.458284 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.458551 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.459165 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.461077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.654557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1120/2223] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:52 2022] POST /identity/v3/users => generated 292 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:52.658804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.660976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:52.673081 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16722c8-84e4-4cdc-a1df-4cc25a0694ce admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.678555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.679236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.679518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.679822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.680470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.682226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.694824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1104/2224] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:52.698503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.700353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.701442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:52.710983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259ced-5bfc-4d9e-ab8b-88e2b7636bb0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.714984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:52.717835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.718448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.720868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.720868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.720868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.721455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.742899 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1105/2225] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:52.752684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:52.765467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05cd9-e114-4fb9-845d-3e5c24f19d6b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.773473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.773473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.773473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.773473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] PATH_INFO: `/v3/projects/c5d537c9bfb24b3b8979f65fca0a50ab/users/73d89275365142b785b291e196d03b54/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.783140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14047b86-fb44-4281-b5e3-3e6ec8f96d6c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:52.787121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb24988-5750-4b1b-8da3-9a616d7e38e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:52.789502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb24988-5750-4b1b-8da3-9a616d7e38e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.789502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb24988-5750-4b1b-8da3-9a616d7e38e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.789502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb24988-5750-4b1b-8da3-9a616d7e38e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.790756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6eb24988-5750-4b1b-8da3-9a616d7e38e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:52.793021 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d537c9bfb24b3b8979f65fca0a50ab, user_id=73d89275365142b785b291e196d03b54, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.796088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.854459 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb24988-5750-4b1b-8da3-9a616d7e38e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:52.856591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb24988-5750-4b1b-8da3-9a616d7e38e6 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:52.865474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1121/2226] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:52 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:52.874595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1106/2227] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:52 2022] PUT /identity/v3/projects/c5d537c9bfb24b3b8979f65fca0a50ab/users/73d89275365142b785b291e196d03b54/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:52.885971 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662984e-0922-42e4-94c1-58de57944afb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.886312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662984e-0922-42e4-94c1-58de57944afb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.886752 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662984e-0922-42e4-94c1-58de57944afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.915009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d662984e-0922-42e4-94c1-58de57944afb None None] MFA Rules not processed for user `73d89275365142b785b291e196d03b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:52.916264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d662984e-0922-42e4-94c1-58de57944afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:52.984198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1122/2228] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:52.995593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:53.054214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac45dcd-e795-4bcd-ad9d-256991d145cb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:53.058056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca62605-1e1c-493a-bfcc-7735f8036213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:53.058728 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62605-1e1c-493a-bfcc-7735f8036213 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:53.059011 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62605-1e1c-493a-bfcc-7735f8036213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:53.059386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62605-1e1c-493a-bfcc-7735f8036213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:53.067861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca62605-1e1c-493a-bfcc-7735f8036213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:53.069350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca62605-1e1c-493a-bfcc-7735f8036213 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:53.078375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1107/2229] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:53.596604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:53.905858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:54.802183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:54.882658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:55.342812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:56.413037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662984e-0922-42e4-94c1-58de57944afb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:56.414482 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d662984e-0922-42e4-94c1-58de57944afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:56.484625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662984e-0922-42e4-94c1-58de57944afb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:56.489066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:56.489690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:56.489968 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:56.490239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] PATH_INFO: `/v3/users/58ed2b7d11a34fa386e22defcab690d8` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:56.491347 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ed2b7d11a34fa386e22defcab690d8)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:56.493293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:56.772896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ed2b7d11a34fa386e22defcab690d8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:56.773440 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ed2b7d11a34fa386e22defcab690d8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:56.777380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:56.864342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ed2b7d11a34fa386e22defcab690d8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:56.864955 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ed2b7d11a34fa386e22defcab690d8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:56.872378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ed2b7d11a34fa386e22defcab690d8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:56.890623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1123/2230] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:56 2022] DELETE /identity/v3/users/58ed2b7d11a34fa386e22defcab690d8 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:56.895497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:56.896058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:56.896274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:56.916727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:56.917927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:56.984098 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1108/2231] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:56.994320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:56.995798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:57.053884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c7d2d-b2cf-4431-b7be-53dae57c6546 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:57.057490 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:57.058090 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:57.058367 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:57.058637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:57.068344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:57.069767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:57.079804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1124/2232] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:57.507329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:57.508766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:57.522715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d603e-a198-41b6-9218-2918c66c37d2 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:57.529235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:57.530022 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:57.530635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:57.531107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] PATH_INFO: `/v3/projects/6ac2712fd0674a4a9104a6a778152ce7` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:57.532358 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac2712fd0674a4a9104a6a778152ce7)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:57.534540 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:57.672753 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac2712fd0674a4a9104a6a778152ce7', 'request_id': 'req-e4c33ae7-9399-41c1-b98d-33a2be4e9621'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:57.677232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:57.679909 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac2712fd0674a4a9104a6a778152ce7', 'request_id': 'req-e4c33ae7-9399-41c1-b98d-33a2be4e9621'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:57.686729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac2712fd0674a4a9104a6a778152ce7', 'request_id': 'req-e4c33ae7-9399-41c1-b98d-33a2be4e9621'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:57.787811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] The token cache is being invalidate because project 6ac2712fd0674a4a9104a6a778152ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:57.788241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:57.789049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:57.790759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1109/2233] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:57 2022] DELETE /identity/v3/projects/6ac2712fd0674a4a9104a6a778152ce7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:57.795021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:57.804800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd34851-a41a-4068-9062-9db4d58a9a27 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:57.859772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:57.860435 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:57.860711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:57.861039 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] PATH_INFO: `/v3/users/a664d8013aa14b29a331b3d7facbd621` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:57.862087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a664d8013aa14b29a331b3d7facbd621)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:57.864071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:58.059694 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a664d8013aa14b29a331b3d7facbd621'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:58.060383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a664d8013aa14b29a331b3d7facbd621'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:58.144783 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a664d8013aa14b29a331b3d7facbd621'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:58.145395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a664d8013aa14b29a331b3d7facbd621'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:58.154111 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b8846a80-aa70-4774-a86e-0d993beffb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a664d8013aa14b29a331b3d7facbd621'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:58.163499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:58.174758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1125/2234] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:12:57 2022] DELETE /identity/v3/users/a664d8013aa14b29a331b3d7facbd621 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:12:58.758484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:58.759981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:58.773777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c33ae7-9399-41c1-b98d-33a2be4e9621 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:58.832617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:58.833182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:58.833493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:58.833768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] PATH_INFO: `/v3/projects/9b2c34d3ec88442a9ac9b6b7c0b23960` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:58.834893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b2c34d3ec88442a9ac9b6b7c0b23960)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:58.836866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:58.947796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b2c34d3ec88442a9ac9b6b7c0b23960', 'request_id': 'req-e5470faf-a778-4bf4-917e-7bce190f24c0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:58.954248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b2c34d3ec88442a9ac9b6b7c0b23960', 'request_id': 'req-e5470faf-a778-4bf4-917e-7bce190f24c0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:58.960563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b2c34d3ec88442a9ac9b6b7c0b23960', 'request_id': 'req-e5470faf-a778-4bf4-917e-7bce190f24c0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:59.079727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] The token cache is being invalidate because project 9b2c34d3ec88442a9ac9b6b7c0b23960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:12:59.080049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:59.080685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e5470faf-a778-4bf4-917e-7bce190f24c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:12:59.082210 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1110/2235] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:12:58 2022] DELETE /identity/v3/projects/9b2c34d3ec88442a9ac9b6b7c0b23960 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:59.087115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fd38c-0423-4720-804e-30a402a0bf5c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.087589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fd38c-0423-4720-804e-30a402a0bf5c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.087589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fd38c-0423-4720-804e-30a402a0bf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.110907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f42fd38c-0423-4720-804e-30a402a0bf5c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:59.112152 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f42fd38c-0423-4720-804e-30a402a0bf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:59.172329 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1126/2236] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:59.180430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ffb96-b718-469b-8597-a363a216de14 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.180720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ffb96-b718-469b-8597-a363a216de14 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.180997 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ffb96-b718-469b-8597-a363a216de14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.202663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-4d1ffb96-b718-469b-8597-a363a216de14 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:59.203908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4d1ffb96-b718-469b-8597-a363a216de14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:59.212102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1111/2237] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:59.217470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fd38c-0423-4720-804e-30a402a0bf5c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.227073 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fd38c-0423-4720-804e-30a402a0bf5c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:59.231719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:59.232348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.232625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.232896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.233533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:12:59.233834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:59.235419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:59.247944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1127/2238] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:12:59.252177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ffb96-b718-469b-8597-a363a216de14 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.261858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ffb96-b718-469b-8597-a363a216de14 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:59.266637 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:59.267230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.267504 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.267771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.268433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:59.270337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:59.393649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1112/2239] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:59 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:12:59.399189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.409872 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccd7cd-9381-4e37-b220-51a4019b0fd9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:59.464867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:59.466245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.466709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.466984 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.467615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:59.469571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:59.594706 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1128/2240] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:12:59 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:12:59.599034 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.610456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca67f0-355e-4e3d-a861-40d7e9b743ec admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:59.615170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:59.615830 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.616110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.616376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.617009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:59.618608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:59.629337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1113/2241] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:59.633682 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.644313 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ab91d-94f6-4ffc-8873-496b80bf621a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:59.648850 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15149794-42ea-4fd1-a04f-6258d3909456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:59.649439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15149794-42ea-4fd1-a04f-6258d3909456 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.649713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15149794-42ea-4fd1-a04f-6258d3909456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.649982 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15149794-42ea-4fd1-a04f-6258d3909456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.650606 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15149794-42ea-4fd1-a04f-6258d3909456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:59.652222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15149794-42ea-4fd1-a04f-6258d3909456 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:59.664129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1129/2242] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:59.668307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.678287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c2c04-5e41-4760-9521-87e1e37e0822 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:59.682944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:59.684615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.684615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.684615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] PATH_INFO: `/v3/projects/fb01e87771e043328bc21d1588988472/users/996e99998ebc4e1ba42f0c14446c7706/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.699174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb01e87771e043328bc21d1588988472, user_id=996e99998ebc4e1ba42f0c14446c7706, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:59.702521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:59.815106 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1114/2243] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:12:59 2022] PUT /identity/v3/projects/fb01e87771e043328bc21d1588988472/users/996e99998ebc4e1ba42f0c14446c7706/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:12:59.830283 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948043c-054c-4bcb-9b12-4845eda53204 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.830594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948043c-054c-4bcb-9b12-4845eda53204 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.830887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948043c-054c-4bcb-9b12-4845eda53204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.862224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6948043c-054c-4bcb-9b12-4845eda53204 None None] MFA Rules not processed for user `996e99998ebc4e1ba42f0c14446c7706`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:59.863507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6948043c-054c-4bcb-9b12-4845eda53204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:59.922330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:59.935304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1130/2244] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:59.939962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.952209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:00.005439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29640f39-8707-4cd3-913a-13b2e17359a1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:00.010411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:00.011053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.011334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.011605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.012995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:00.015031 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:00.176699 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1115/2245] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:12:59 2022] POST /identity/v3/projects => generated 387 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:00.180785 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948043c-054c-4bcb-9b12-4845eda53204 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:00.191902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948043c-054c-4bcb-9b12-4845eda53204 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:00.246091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:00.247414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.247672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.247967 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.248599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:00.250525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:00.387751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:00.388323 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1131/2246] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:00 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:00.399566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027e368-7dae-4076-bf36-62278c02bb58 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:00.404522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:00.405124 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.405404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.405731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.406417 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:00.408136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:00.420258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1116/2247] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:13:00.424445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:00.435778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577d5d-505f-4939-970e-a7344ccef71c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:00.440394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194d946-8aa5-4712-b92b-3f97a5b20348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:00.440987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194d946-8aa5-4712-b92b-3f97a5b20348 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.441264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194d946-8aa5-4712-b92b-3f97a5b20348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.441536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194d946-8aa5-4712-b92b-3f97a5b20348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.442233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194d946-8aa5-4712-b92b-3f97a5b20348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:00.443869 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194d946-8aa5-4712-b92b-3f97a5b20348 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:00.454673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1132/2248] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:00.459056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:00.470213 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f5eb7-4e37-4074-89c6-b3bd6ccbcde5 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:00.474848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:00.478597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.478597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.478597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] PATH_INFO: `/v3/projects/55b1ed0c7b8341da918970afb6783c61/users/e276ed9fea284f3589032df8f877d583/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.493717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b1ed0c7b8341da918970afb6783c61, user_id=e276ed9fea284f3589032df8f877d583, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:00.496525 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:00.584108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1117/2249] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:00 2022] PUT /identity/v3/projects/55b1ed0c7b8341da918970afb6783c61/users/e276ed9fea284f3589032df8f877d583/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:00.597331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab6f40-51db-433b-b231-dee93fee9f95 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.597616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab6f40-51db-433b-b231-dee93fee9f95 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.597920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab6f40-51db-433b-b231-dee93fee9f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.628467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5aab6f40-51db-433b-b231-dee93fee9f95 None None] MFA Rules not processed for user `e276ed9fea284f3589032df8f877d583`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:00.629604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5aab6f40-51db-433b-b231-dee93fee9f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:00.695296 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1133/2250] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:00.700233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:00.761172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3d276-5ced-4519-8e94-232fb3c324da admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:00.765903 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:00.766502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.766781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.767053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.767672 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:00.769615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:00.864214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1118/2251] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:00 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:00.868565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab6f40-51db-433b-b231-dee93fee9f95 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:00.879918 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab6f40-51db-433b-b231-dee93fee9f95 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:00.929944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:00.936641 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:00.937248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:00.937528 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:00.937842 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:00.938474 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:00.940439 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:01.021775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1134/2252] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:00 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:01.025878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:01.036975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb785b-21a4-411d-a2a3-7f5464718696 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:01.042404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:01.042726 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.043042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.043329 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:01.043953 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:01.045592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:01.056741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1119/2253] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:01.061268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:01.071266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3280187-892a-423e-9640-b7044b4344d2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:01.076702 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:01.077350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.077628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.077896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] PATH_INFO: `/v3/projects/6f53cb45bcb44873a53f257162a07fde/users/ac7f6d9946164f9ababbae19e8beb75e/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:01.093173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f53cb45bcb44873a53f257162a07fde, user_id=ac7f6d9946164f9ababbae19e8beb75e, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:01.096360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:01.172058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1135/2254] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:01 2022] PUT /identity/v3/projects/6f53cb45bcb44873a53f257162a07fde/users/ac7f6d9946164f9ababbae19e8beb75e/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:01.176087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:01.186337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dc4b7-0f32-4215-9801-afee5dda4ecc admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:01.218901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:01.246276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335a0e6-b87f-4569-b035-205235f224e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:01.247545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a0e6-b87f-4569-b035-205235f224e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.247998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a0e6-b87f-4569-b035-205235f224e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.248273 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a0e6-b87f-4569-b035-205235f224e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:01.248914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335a0e6-b87f-4569-b035-205235f224e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:01.251110 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335a0e6-b87f-4569-b035-205235f224e3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:01.261965 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1120/2255] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:01.266956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:01.278887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158dbb0-ee32-4bad-a33e-6366f9434538 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:01.283636 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:01.284359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.284644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.284921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] PATH_INFO: `/v3/projects/6f53cb45bcb44873a53f257162a07fde/users/ac7f6d9946164f9ababbae19e8beb75e/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:01.286858 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f53cb45bcb44873a53f257162a07fde, user_id=ac7f6d9946164f9ababbae19e8beb75e, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:01.289950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:01.301352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1136/2256] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:01 2022] PUT /identity/v3/projects/6f53cb45bcb44873a53f257162a07fde/users/ac7f6d9946164f9ababbae19e8beb75e/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:01.312808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.313183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.313536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:01.343949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] MFA Rules not processed for user `ac7f6d9946164f9ababbae19e8beb75e`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:01.345246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:01.422247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1121/2257] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:01.433493 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:01.436467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:01.499672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d4c2b-b04e-4e69-9512-27f6a806c30e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:01.503415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:01.504044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.504324 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.504597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:01.514973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:01.516548 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:01.527054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1137/2258] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:02.580145 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:02.581980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:02.671433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8679872-b85b-49b1-aec9-ee24130c958c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:02.676005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:02.676667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:02.676946 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:02.677220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] PATH_INFO: `/v3/users/73d89275365142b785b291e196d03b54` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:02.678500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d89275365142b785b291e196d03b54)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:02.680477 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:02.894186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d89275365142b785b291e196d03b54'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:02.901863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:02.908205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d89275365142b785b291e196d03b54'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:02.908931 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d89275365142b785b291e196d03b54'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:02.915752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d89275365142b785b291e196d03b54'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:02.918236 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bca85b03-33c3-466f-b20a-9a317e15f134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d89275365142b785b291e196d03b54'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:02.946216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:02.952370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:02.962436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4b57d-43c7-40f9-a7c3-99047363589d service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:02.966952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:02.967578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:02.967884 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:02.968160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] PATH_INFO: `/v3/users/9f850368cfb0451fa12d2a2626562f5c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:02.969266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f850368cfb0451fa12d2a2626562f5c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:02.971166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:03.067638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1122/2259] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:02 2022] DELETE /identity/v3/users/73d89275365142b785b291e196d03b54 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:03.072015 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:03.072336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:03.073128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:03.094448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:03.095761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:03.163635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1123/2260] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:03.175231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:03.177335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:03.239838 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a791a-553f-4fee-b5c7-910fe9c70cef None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:03.243594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:03.244298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:03.248237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:03.248237 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:03.248916 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f850368cfb0451fa12d2a2626562f5c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:03.249569 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f850368cfb0451fa12d2a2626562f5c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:03.256272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:03.258650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:03.269820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1124/2261] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:13:03.270592 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:03.271874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:03.332854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c594fb-9daa-4f6b-92f5-651ef7b8f1b2 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:03.336996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:03.337967 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:03.338307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:03.338715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:03.341246 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:03.343185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f850368cfb0451fa12d2a2626562f5c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:03.344112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f850368cfb0451fa12d2a2626562f5c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:03.352671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-739e439a-719c-43ad-8187-3a74db799389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f850368cfb0451fa12d2a2626562f5c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:03.375625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1138/2262] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:02 2022] DELETE /identity/v3/users/9f850368cfb0451fa12d2a2626562f5c => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:03.379902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:03.388059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:03.388059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:03.388317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:03.404813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:03.406042 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:03.414342 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:03.415781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:03.426016 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1125/2263] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:03.427183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:03.479828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1139/2264] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:03.488550 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:03.489974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:03.504594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75f8dd-a533-4b8a-9b62-690f8f0d75cb service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:03.570509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ab270-75d7-4675-b425-d994488224d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:03.570509 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab270-75d7-4675-b425-d994488224d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:03.570766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab270-75d7-4675-b425-d994488224d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:03.570927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ab270-75d7-4675-b425-d994488224d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:03.581485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ab270-75d7-4675-b425-d994488224d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:03.583891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ab270-75d7-4675-b425-d994488224d7 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:03.595283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1126/2265] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:03.784743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:03.786579 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:03.802143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c90a94-e19d-4524-9fd8-c3e4380eab8d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:03.808574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:03.809441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:03.810242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:03.810242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] PATH_INFO: `/v3/projects/c5d537c9bfb24b3b8979f65fca0a50ab` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:03.812354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d537c9bfb24b3b8979f65fca0a50ab)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:03.814420 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.011550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d537c9bfb24b3b8979f65fca0a50ab', 'request_id': 'req-86ba6d68-3c8a-49a9-abc4-1f576448715f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.020067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d537c9bfb24b3b8979f65fca0a50ab', 'request_id': 'req-86ba6d68-3c8a-49a9-abc4-1f576448715f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.039257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d537c9bfb24b3b8979f65fca0a50ab', 'request_id': 'req-86ba6d68-3c8a-49a9-abc4-1f576448715f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.231985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] The token cache is being invalidate because project c5d537c9bfb24b3b8979f65fca0a50ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:04.231985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.231985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86ba6d68-3c8a-49a9-abc4-1f576448715f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.231985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1140/2266] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:03 2022] DELETE /identity/v3/projects/c5d537c9bfb24b3b8979f65fca0a50ab => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:04.232467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.232467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.232467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.252692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:04.255028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:04.315049 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1127/2267] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:04.323151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413821-78a1-40e0-89d0-049a7a4431bf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.323450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413821-78a1-40e0-89d0-049a7a4431bf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.324231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413821-78a1-40e0-89d0-049a7a4431bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.361427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9b413821-78a1-40e0-89d0-049a7a4431bf None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:04.361427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9b413821-78a1-40e0-89d0-049a7a4431bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:04.362676 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.365174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:04.368988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1141/2268] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:04.375907 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b413821-78a1-40e0-89d0-049a7a4431bf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.393658 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84090353-88fa-4349-acad-70f9d7a4096a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.393887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b413821-78a1-40e0-89d0-049a7a4431bf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.394582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.395220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.395500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.395928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] PATH_INFO: `/v3/projects/cc8d93270e4044079799e8cfa1882ac4` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.396580 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.397195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.397323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8d93270e4044079799e8cfa1882ac4)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.397467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.397903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.398575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:13:04.398873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.399169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.400614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.423429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1142/2269] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:13:04.430315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.440996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96ea0-1845-48ad-a097-46363bb0883e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.445637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.446231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.446513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.446781 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.447401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.449495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.525798 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8d93270e4044079799e8cfa1882ac4', 'request_id': 'req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.531528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8d93270e4044079799e8cfa1882ac4', 'request_id': 'req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.537963 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8d93270e4044079799e8cfa1882ac4', 'request_id': 'req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.544236 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1143/2270] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:04 2022] POST /identity/v3/projects => generated 397 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:04.549306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.622906 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c97f-3395-411a-976e-bfa439442911 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.627742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.628384 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.628665 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.628944 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.629196 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:04.629713 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.631500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.634567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] The token cache is being invalidate because project cc8d93270e4044079799e8cfa1882ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:04.634862 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.635597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.637313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1128/2271] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:04 2022] DELETE /identity/v3/projects/cc8d93270e4044079799e8cfa1882ac4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:04.641388 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.642305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:04.642868 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:04.709575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78a00e-3c2a-4402-855e-7ab59c51e3ca admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.714506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.715144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.715526 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.715827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] PATH_INFO: `/v3/users/dacc4a458cc94640ac93c999679f41ae` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.717073 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dacc4a458cc94640ac93c999679f41ae)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.719386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.742237 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1144/2272] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:04 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:04.745177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.747653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:04.760391 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eb756-cf83-4677-948e-18f6b3607d8a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.765277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.768562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.768562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.768562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.768562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.768936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.778886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1145/2273] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:13:04.782753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.794756 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0713b-45a3-458a-91f6-4746d4a6327e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.799549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.800174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.800455 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.800728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.801354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.802973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.814248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1146/2274] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:04.818195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:04.827597 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21b520-5f5a-48eb-bc27-6f1adf2b2780 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:04.832406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9153ee-0c16-406f-863e-1bd2efa2ebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:04.833034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9153ee-0c16-406f-863e-1bd2efa2ebce admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.833312 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9153ee-0c16-406f-863e-1bd2efa2ebce admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.833590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9153ee-0c16-406f-863e-1bd2efa2ebce admin admin] PATH_INFO: `/v3/projects/a11ecc186aa04a4ea292cc31eac3b4fe/users/88e02635bf5b434fb510ea65a92eac58/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.848502 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9153ee-0c16-406f-863e-1bd2efa2ebce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11ecc186aa04a4ea292cc31eac3b4fe, user_id=88e02635bf5b434fb510ea65a92eac58, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:04.851235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9153ee-0c16-406f-863e-1bd2efa2ebce admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:04.927913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dacc4a458cc94640ac93c999679f41ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.941052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dacc4a458cc94640ac93c999679f41ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.946405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dacc4a458cc94640ac93c999679f41ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.948695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1147/2275] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:04 2022] PUT /identity/v3/projects/a11ecc186aa04a4ea292cc31eac3b4fe/users/88e02635bf5b434fb510ea65a92eac58/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:04.958649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dacc4a458cc94640ac93c999679f41ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.959276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.959416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dacc4a458cc94640ac93c999679f41ae'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:04.959741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.960170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.991921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] MFA Rules not processed for user `88e02635bf5b434fb510ea65a92eac58`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:04.993182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:05.069927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1148/2276] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:05.081483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:05.083036 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:05.110234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1129/2277] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:04 2022] DELETE /identity/v3/users/dacc4a458cc94640ac93c999679f41ae => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:05.151185 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50958b-35a5-425a-bd35-0bc2dd3e6cc2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:05.213125 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:05.213730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:05.214004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:05.214271 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:05.220220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:05.273030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:05.274441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:05.284757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1149/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:05 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:13:05.634188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:05.638908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:05.703982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c75136-5711-46f7-9271-ce4de72943c7 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:05.708872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:05.709586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:05.709891 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:05.710169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] PATH_INFO: `/v3/projects/5e1e88b54e41413a951037b176f9f4e0` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:05.711313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1e88b54e41413a951037b176f9f4e0)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:05.713307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:05.737578 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:05.874836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1e88b54e41413a951037b176f9f4e0', 'request_id': 'req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:05.880643 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1e88b54e41413a951037b176f9f4e0', 'request_id': 'req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:05.887092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1e88b54e41413a951037b176f9f4e0', 'request_id': 'req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.002505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] The token cache is being invalidate because project 5e1e88b54e41413a951037b176f9f4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:06.002794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.003437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.004944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1130/2279] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:05 2022] DELETE /identity/v3/projects/5e1e88b54e41413a951037b176f9f4e0 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:06.009401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:06.021761 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec124688-f363-4c5f-97e9-975b78b31b95 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:06.091512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:06.092182 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:06.092484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:06.092755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] PATH_INFO: `/v3/users/ad778be5afe64451b644f6a3a2557b86` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:06.093862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad778be5afe64451b644f6a3a2557b86)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:06.095767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:06.330027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad778be5afe64451b644f6a3a2557b86'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.330692 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad778be5afe64451b644f6a3a2557b86'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.416319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad778be5afe64451b644f6a3a2557b86'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.416993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad778be5afe64451b644f6a3a2557b86'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.425688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad778be5afe64451b644f6a3a2557b86'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:06.443549 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1150/2280] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:06 2022] DELETE /identity/v3/users/ad778be5afe64451b644f6a3a2557b86 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:06.862002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:06.863532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:06.882408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2008a1-b64d-428b-91dc-6ef5ed1dbd1d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:06.954114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:06.954721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:06.955003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:06.955278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] PATH_INFO: `/v3/users/237af7a747604f83b59e3aa8c4a240f1` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:06.956736 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=237af7a747604f83b59e3aa8c4a240f1)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:06.958717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:06.966234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:06.967664 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:07.000910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b6551-2703-454e-bbbc-21e76218af60 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:07.007225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:07.008197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.008508 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.008871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] PATH_INFO: `/v3/projects/1899aceada474c73ac7df7ac03e5ca59` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.010253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1899aceada474c73ac7df7ac03e5ca59)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:07.012671 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:07.057173 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:07.169955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237af7a747604f83b59e3aa8c4a240f1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.186825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1899aceada474c73ac7df7ac03e5ca59', 'request_id': 'req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.193097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1899aceada474c73ac7df7ac03e5ca59', 'request_id': 'req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.200398 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1899aceada474c73ac7df7ac03e5ca59', 'request_id': 'req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.214234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237af7a747604f83b59e3aa8c4a240f1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.215241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237af7a747604f83b59e3aa8c4a240f1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.230115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237af7a747604f83b59e3aa8c4a240f1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.230904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a3260788-39d9-4d1e-8e85-cf7c1cca4bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237af7a747604f83b59e3aa8c4a240f1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.334956 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] The token cache is being invalidate because project 1899aceada474c73ac7df7ac03e5ca59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:07.335316 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.344817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3aa9ba7-d863-4444-a7e4-2f478f9883c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:07.344817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1151/2281] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:06 2022] DELETE /identity/v3/projects/1899aceada474c73ac7df7ac03e5ca59 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:07.346993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37a7de-1248-4086-b70a-1abb0bf384cf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.347669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37a7de-1248-4086-b70a-1abb0bf384cf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.348146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37a7de-1248-4086-b70a-1abb0bf384cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.371231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-dc37a7de-1248-4086-b70a-1abb0bf384cf None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:07.373581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dc37a7de-1248-4086-b70a-1abb0bf384cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:07.414704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1131/2282] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:06 2022] DELETE /identity/v3/users/237af7a747604f83b59e3aa8c4a240f1 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:07.423666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.424297 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.424651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.441647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1152/2283] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:07.446684 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:07.448055 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:07.457813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5e45-4cac-42d3-bc6b-06ed95a2ab01 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.458100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5e45-4cac-42d3-bc6b-06ed95a2ab01 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.458363 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5e45-4cac-42d3-bc6b-06ed95a2ab01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.482445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-65ae5e45-4cac-42d3-bc6b-06ed95a2ab01 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:07.483890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-65ae5e45-4cac-42d3-bc6b-06ed95a2ab01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:07.489651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:07.545264 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1132/2284] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:07.555476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:07.557174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1153/2285] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:07.560720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:07.562532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5e45-4cac-42d3-bc6b-06ed95a2ab01 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:07.572574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5e45-4cac-42d3-bc6b-06ed95a2ab01 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:07.578684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:07.579133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.579470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.579836 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.580572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:13:07.580958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:07.582615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:07.598327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1154/2286] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:13:07.599589 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:07.615827 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641eb7f4-31eb-431e-a736-627b8adb7d65 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:07.620762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:07.621447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.622585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.622979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.623732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:07.625747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:07.648455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b0bc8-d50f-4a26-9e2e-d09083a43d9f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:07.652308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:07.653690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.653690 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.653913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.663599 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:07.665096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:07.677923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1133/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:13:07.759470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1155/2288] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:07 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:07.764932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:07.775140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f3f54-6ee0-4050-9bd7-ed5443685e2b service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:07.833598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:07.843744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.843744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.843744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.843744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:07.843744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:07.941831 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1134/2289] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:07 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:07.950825 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:07.961335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28077823-d800-45c0-9eb8-d41dd1ba5223 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:07.971414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:07.972373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:07.972778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:07.973064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:07.973853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:07.975511 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:07.989988 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1156/2290] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:07.991204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:08.002546 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d99ed2-3c32-4fb7-93ca-c00973b214d0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:08.009791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:08.011927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.012755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.013039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.013797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:08.015576 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:08.039003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1135/2291] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:08.042522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:08.051231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:08.062354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcedcd-e882-473c-ba91-2952cff40924 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:08.074442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab01608-cd0e-47a5-8d0c-eedbff003864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:08.075709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01608-cd0e-47a5-8d0c-eedbff003864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.076268 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01608-cd0e-47a5-8d0c-eedbff003864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.076698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01608-cd0e-47a5-8d0c-eedbff003864 admin admin] PATH_INFO: `/v3/projects/ae8110e8bac044f8a492065e1e06a48d/users/24bee4c00fab42bd8b0cf916e491890d/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.095647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab01608-cd0e-47a5-8d0c-eedbff003864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae8110e8bac044f8a492065e1e06a48d, user_id=24bee4c00fab42bd8b0cf916e491890d, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:08.098683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab01608-cd0e-47a5-8d0c-eedbff003864 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:08.196281 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:08.201020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1157/2292] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:08 2022] PUT /identity/v3/projects/ae8110e8bac044f8a492065e1e06a48d/users/24bee4c00fab42bd8b0cf916e491890d/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:08.201951 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:08.220898 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfac1a7-75c8-4a8b-996f-dd45f6b38f4f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:08.221614 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.223066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.223630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.265540 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:08.266767 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:08.295303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:08.296590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.296590 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.302074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] PATH_INFO: `/v3/projects/404e1b1ea8974c95a038f43fda656503` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.302074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=404e1b1ea8974c95a038f43fda656503)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:08.302074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:08.331028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1158/2293] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:08.341443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:08.342883 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:08.404319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad91c-cff6-4be3-bb77-fc69a8d80ab8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:08.407987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:08.408593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.408867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.409135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.418317 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:08.419866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:08.429159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1159/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:13:08.477529 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '404e1b1ea8974c95a038f43fda656503', 'request_id': 'req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:08.482932 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '404e1b1ea8974c95a038f43fda656503', 'request_id': 'req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:08.489147 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '404e1b1ea8974c95a038f43fda656503', 'request_id': 'req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:08.568455 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:08.609152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] The token cache is being invalidate because project 404e1b1ea8974c95a038f43fda656503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:08.609543 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:08.610277 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:08.612052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1136/2295] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:08 2022] DELETE /identity/v3/projects/404e1b1ea8974c95a038f43fda656503 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:08.616217 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:08.642156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b3322-ee50-49c6-877c-30a7f5a322fc service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:08.706875 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:08.707417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.707694 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.708021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] PATH_INFO: `/v3/users/4e9c5df2f8314b7abc2291cea86bdb5f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.709104 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9c5df2f8314b7abc2291cea86bdb5f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:08.711051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:08.985739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9c5df2f8314b7abc2291cea86bdb5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:08.987536 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9c5df2f8314b7abc2291cea86bdb5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:09.060498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9c5df2f8314b7abc2291cea86bdb5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:09.061285 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9c5df2f8314b7abc2291cea86bdb5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:09.069689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b2f299da-f01e-4890-be08-fec03790464e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9c5df2f8314b7abc2291cea86bdb5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:09.087331 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1160/2296] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:08 2022] DELETE /identity/v3/users/4e9c5df2f8314b7abc2291cea86bdb5f => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:09.516996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:09.647305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:09.648848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:09.662434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2ee3e-fcd8-4f38-99ed-a01bf7005aa2 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:09.719212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:09.719876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:09.720159 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:09.727948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] PATH_INFO: `/v3/projects/4814a58897e342a4b37b7c121e191f26` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:09.727948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4814a58897e342a4b37b7c121e191f26)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:09.727948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:09.886430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4814a58897e342a4b37b7c121e191f26', 'request_id': 'req-42ac6361-9b76-469b-833c-3e1cd0eab9e1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:09.892524 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4814a58897e342a4b37b7c121e191f26', 'request_id': 'req-42ac6361-9b76-469b-833c-3e1cd0eab9e1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:09.899180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4814a58897e342a4b37b7c121e191f26', 'request_id': 'req-42ac6361-9b76-469b-833c-3e1cd0eab9e1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:10.009840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] The token cache is being invalidate because project 4814a58897e342a4b37b7c121e191f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:10.009840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:10.009840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-42ac6361-9b76-469b-833c-3e1cd0eab9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:10.011141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1137/2297] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:09 2022] DELETE /identity/v3/projects/4814a58897e342a4b37b7c121e191f26 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:10.015921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1cda-4679-4d51-b657-28dea4df11eb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.016213 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1cda-4679-4d51-b657-28dea4df11eb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.016492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1cda-4679-4d51-b657-28dea4df11eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.046603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7e5d1cda-4679-4d51-b657-28dea4df11eb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:10.047901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7e5d1cda-4679-4d51-b657-28dea4df11eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:10.108219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1161/2298] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:10.117484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311dff8-9f4a-4258-a03a-50573289e27b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.117771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311dff8-9f4a-4258-a03a-50573289e27b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.118053 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311dff8-9f4a-4258-a03a-50573289e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.142175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-3311dff8-9f4a-4258-a03a-50573289e27b None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:10.143395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-3311dff8-9f4a-4258-a03a-50573289e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:10.154939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1138/2299] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:10.157763 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1cda-4679-4d51-b657-28dea4df11eb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:10.168227 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1cda-4679-4d51-b657-28dea4df11eb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:10.173861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:10.180586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.180586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.180586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.180586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:13:10.180586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:10.180586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:10.194000 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1162/2300] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:13:10.195459 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311dff8-9f4a-4258-a03a-50573289e27b None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:10.200101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:10.205336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311dff8-9f4a-4258-a03a-50573289e27b None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:10.210138 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:10.210740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.211017 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.211283 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.211939 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:10.213840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:10.373539 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1139/2301] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:10 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:10.378461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:10.389483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81609ac7-945c-4858-8ed0-5a04f64dda10 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:10.448098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:10.448705 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.448978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.449246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.449861 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:10.452531 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:10.538871 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1163/2302] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:10 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:10.542389 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:10.554334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc4f05-8b12-4319-8252-3198f858fae6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:10.559855 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:10.560468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.560745 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.561014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.563835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:10.563835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:10.574854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1140/2303] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:10.578728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:10.589288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae8546-7456-46d3-b526-ad891fac995a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:10.596862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426d9b7-af25-4419-a7fc-e53206e490b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:10.602311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d9b7-af25-4419-a7fc-e53206e490b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.602311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d9b7-af25-4419-a7fc-e53206e490b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.602311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d9b7-af25-4419-a7fc-e53206e490b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.602311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426d9b7-af25-4419-a7fc-e53206e490b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:10.602311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426d9b7-af25-4419-a7fc-e53206e490b1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:10.613143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1164/2304] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:10.617885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:10.632350 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05cb58-62ca-4a34-a73f-231fcf017608 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:10.637190 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:10.639201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.639608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.639924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] PATH_INFO: `/v3/projects/0ac7469cc0804ddb9702159dddc67b39/users/ce2972d552564baca6fa686b84ff45ba/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.656949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac7469cc0804ddb9702159dddc67b39, user_id=ce2972d552564baca6fa686b84ff45ba, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:10.659611 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:10.737570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1141/2305] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:10 2022] PUT /identity/v3/projects/0ac7469cc0804ddb9702159dddc67b39/users/ce2972d552564baca6fa686b84ff45ba/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:10.742769 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.743056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.743326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.763139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:10.764790 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:10.859927 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1165/2306] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:10.869727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:10.871573 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:10.941430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b421be-8f39-4d3a-b25b-47564d8dc550 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:10.946447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec45afb-33f6-46ad-9590-68f7285aff9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:10.947071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec45afb-33f6-46ad-9590-68f7285aff9b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:10.947353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec45afb-33f6-46ad-9590-68f7285aff9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:10.947642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec45afb-33f6-46ad-9590-68f7285aff9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:10.963560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec45afb-33f6-46ad-9590-68f7285aff9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:10.963560 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec45afb-33f6-46ad-9590-68f7285aff9b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:10.971629 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1142/2307] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:11.025733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:12.465294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:12.467135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:12.574191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996c603-3cba-4a0a-bace-7e666df9520f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:12.579690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:12.580464 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:12.580691 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:12.581079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:12.596046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:12.753396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:12.755239 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:12.769738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1166/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:12.841475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35a4c-40c5-4c33-93c3-fd88cf1af41d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:12.841475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35a4c-40c5-4c33-93c3-fd88cf1af41d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:12.841475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35a4c-40c5-4c33-93c3-fd88cf1af41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:12.882443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b7e35a4c-40c5-4c33-93c3-fd88cf1af41d None None] MFA Rules not processed for user `ce2972d552564baca6fa686b84ff45ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:12.884652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b7e35a4c-40c5-4c33-93c3-fd88cf1af41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:12.971433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1143/2309] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:12.975970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:12.989299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f09d58-7240-4cb1-8acc-eb125a570088 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:12.995211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:12.995945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:12.996293 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:12.996637 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:12.997937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:13.000177 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:13.112192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1167/2310] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:12 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:13.113130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e35a4c-40c5-4c33-93c3-fd88cf1af41d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:13.128343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e35a4c-40c5-4c33-93c3-fd88cf1af41d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:13.174450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:13.208356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:13.208988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:13.209279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:13.209625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:13.210475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:13.212643 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:13.373234 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1144/2311] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:13 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:13.377457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:13.390568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5f476-eecf-4ee5-8d98-64aca886ce76 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:13.395949 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:13.396557 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:13.396965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:13.397244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:13.398235 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:13.399920 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:13.414678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1168/2312] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:13.419425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:13.429625 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a283ab-b6cf-417c-bed9-f653ea394d45 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:13.435410 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:13.436123 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:13.436403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:13.436683 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] PATH_INFO: `/v3/projects/39f7d996ceea495995116b3ccceb0c40/users/e48eb00bef4f49d9b2cdc2ca12dba22c/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:13.458670 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f7d996ceea495995116b3ccceb0c40, user_id=e48eb00bef4f49d9b2cdc2ca12dba22c, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:13.461766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:13.557968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1145/2313] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:13 2022] PUT /identity/v3/projects/39f7d996ceea495995116b3ccceb0c40/users/e48eb00bef4f49d9b2cdc2ca12dba22c/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:13.562304 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:13.573338 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633195-fdad-4812-831e-ba98f16d8d85 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:13.610623 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:13.640951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0542d64-ae3f-45d1-8bc9-751218c74b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:13.641596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0542d64-ae3f-45d1-8bc9-751218c74b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:13.641874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0542d64-ae3f-45d1-8bc9-751218c74b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:13.642147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0542d64-ae3f-45d1-8bc9-751218c74b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:13.642868 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0542d64-ae3f-45d1-8bc9-751218c74b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:13.644495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0542d64-ae3f-45d1-8bc9-751218c74b87 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:13.655011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1169/2314] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:13.659248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:13.670627 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a37c-fe0c-49c1-a936-ef0096cd2243 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:13.675591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:13.676296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:13.676615 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:13.676886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] PATH_INFO: `/v3/projects/39f7d996ceea495995116b3ccceb0c40/users/e48eb00bef4f49d9b2cdc2ca12dba22c/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:13.678808 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f7d996ceea495995116b3ccceb0c40, user_id=e48eb00bef4f49d9b2cdc2ca12dba22c, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:13.681587 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:13.696552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1146/2315] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:13 2022] PUT /identity/v3/projects/39f7d996ceea495995116b3ccceb0c40/users/e48eb00bef4f49d9b2cdc2ca12dba22c/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:14.976757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:14.976757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:14.976757 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.017305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] MFA Rules not processed for user `24bee4c00fab42bd8b0cf916e491890d`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:15.017305 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:15.149465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:15.154205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:15.154436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1170/2316] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:15.155533 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:15.166571 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:15.269320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35427e15-fb16-43c9-b5c1-e259f880569d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:15.274271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:15.274921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:15.275202 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:15.275432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310eea4-247e-4525-822b-bae17f21d9a4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:15.275541 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] PATH_INFO: `/v3/users/88e02635bf5b434fb510ea65a92eac58` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.276693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e02635bf5b434fb510ea65a92eac58)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:15.278816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:15.296595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:15.296595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:15.296595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:15.296595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.297214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:15.297214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:15.420887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1171/2317] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:15 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:15.424945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:15.500893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae431c-d789-470f-824b-b25c79eb4c92 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:15.505992 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:15.506619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:15.510993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:15.510993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.510993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:15.510993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:15.586660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e02635bf5b434fb510ea65a92eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:15.603748 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e02635bf5b434fb510ea65a92eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:15.604777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e02635bf5b434fb510ea65a92eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:15.613203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e02635bf5b434fb510ea65a92eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:15.614096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e02635bf5b434fb510ea65a92eac58'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:15.633587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1172/2318] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:15 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:15.634445 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:15.636500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:15.651996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfbca7-deed-4adf-850d-a1688ad68c80 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:15.657766 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:15.659139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:15.659524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:15.659957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] PATH_INFO: `/v3/users/ce7478c14d9245b9bc75a63af96231d9` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.661448 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7478c14d9245b9bc75a63af96231d9)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:15.664143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:15.734814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1147/2319] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:15 2022] DELETE /identity/v3/users/88e02635bf5b434fb510ea65a92eac58 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:15.735679 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:15.737251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:15.764641 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:15.827405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f5a12-86d2-4be5-b8e2-4d4fe0e4976b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:15.832338 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cc8fb-d799-4d94-8dc4-755015ca454d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:15.832781 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc8fb-d799-4d94-8dc4-755015ca454d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:15.833056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc8fb-d799-4d94-8dc4-755015ca454d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:15.833328 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc8fb-d799-4d94-8dc4-755015ca454d admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.833981 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cc8fb-d799-4d94-8dc4-755015ca454d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:15.835746 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cc8fb-d799-4d94-8dc4-755015ca454d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:15.846127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:15.855042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1148/2320] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:15.855042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4046a-ca66-4e4a-b9f3-5a881a292c7b None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:15.855042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4046a-ca66-4e4a-b9f3-5a881a292c7b None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:15.855042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4046a-ca66-4e4a-b9f3-5a881a292c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.886833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d2c4046a-ca66-4e4a-b9f3-5a881a292c7b None None] MFA Rules not processed for user `e48eb00bef4f49d9b2cdc2ca12dba22c`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:15.888962 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d2c4046a-ca66-4e4a-b9f3-5a881a292c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:15.922726 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7478c14d9245b9bc75a63af96231d9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:15.923489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7478c14d9245b9bc75a63af96231d9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:15.967111 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1149/2321] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:15.968179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cace5-732a-4fbc-9e1c-088ac55694c4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:15.968464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cace5-732a-4fbc-9e1c-088ac55694c4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:15.968729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cace5-732a-4fbc-9e1c-088ac55694c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:15.989014 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-714cace5-732a-4fbc-9e1c-088ac55694c4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:15.990316 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-714cace5-732a-4fbc-9e1c-088ac55694c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:15.997147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7478c14d9245b9bc75a63af96231d9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:16.000957 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7478c14d9245b9bc75a63af96231d9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:16.001068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1150/2322] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:16.001068 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cace5-732a-4fbc-9e1c-088ac55694c4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:16.010950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cace5-732a-4fbc-9e1c-088ac55694c4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:16.015839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7478c14d9245b9bc75a63af96231d9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:16.016821 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:16.017569 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:16.018345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:16.018345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] PATH_INFO: `/v3/projects/3782de7bdb2147809bc30f526734ea37/users/a0312b8d61eb456c989ba3f57018e38c/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:16.038796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3782de7bdb2147809bc30f526734ea37, user_id=a0312b8d61eb456c989ba3f57018e38c, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:16.041968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:16.047203 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1173/2323] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:15 2022] DELETE /identity/v3/users/ce7478c14d9245b9bc75a63af96231d9 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:16.049214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:16.051106 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:16.127020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1151/2324] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:15 2022] PUT /identity/v3/projects/3782de7bdb2147809bc30f526734ea37/users/a0312b8d61eb456c989ba3f57018e38c/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:16.127651 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:16.131464 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:16.137359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5cbed-4922-4b4e-b529-9632dc5624e2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:16.212085 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98966d9f-b766-4586-b338-f9873ce518b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:16.212712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966d9f-b766-4586-b338-f9873ce518b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:16.212998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966d9f-b766-4586-b338-f9873ce518b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:16.213262 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966d9f-b766-4586-b338-f9873ce518b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:16.220132 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-98966d9f-b766-4586-b338-f9873ce518b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:16.228776 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c5ea3-063e-4a6b-aff0-26b395232c6b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:16.233902 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:16.234536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:16.234819 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:16.235099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:16.237244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:16.333979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98966d9f-b766-4586-b338-f9873ce518b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:16.338738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98966d9f-b766-4586-b338-f9873ce518b9 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:16.338874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:16.338874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:16.347443 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1174/2325] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:16.349038 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ac10a-9447-4201-af69-09544f6c79df None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:16.349599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ac10a-9447-4201-af69-09544f6c79df None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:16.349898 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ac10a-9447-4201-af69-09544f6c79df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:16.364584 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1152/2326] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:16.365481 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:16.367434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:16.374919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-493ac10a-9447-4201-af69-09544f6c79df None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:16.376216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-493ac10a-9447-4201-af69-09544f6c79df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:16.384603 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262e0d9-fb9e-4b0f-b001-892970b90eff service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:16.385937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1175/2327] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:16.397044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ac10a-9447-4201-af69-09544f6c79df None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:16.398212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:16.398762 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-493ac10a-9447-4201-af69-09544f6c79df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:16.399695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:16.400315 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:16.400751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:16.401729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:16.405823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:16.425027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ac10a-9447-4201-af69-09544f6c79df None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:16.425240 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1153/2328] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:16.426711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:16.432990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:16.433610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:16.433886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:16.434156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:16.449449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bf0a5-7bbd-4a89-b12c-227abc1c5222 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:16.452117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:16.453714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:16.455661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:16.456438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:16.457029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:16.457029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] PATH_INFO: `/v3/projects/3782de7bdb2147809bc30f526734ea37/users/a0312b8d61eb456c989ba3f57018e38c/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:16.459539 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3782de7bdb2147809bc30f526734ea37, user_id=a0312b8d61eb456c989ba3f57018e38c, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:16.462885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:16.483771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1154/2329] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:16 2022] PUT /identity/v3/projects/3782de7bdb2147809bc30f526734ea37/users/a0312b8d61eb456c989ba3f57018e38c/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:16.483990 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1176/2330] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:13:16.919751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:16.922888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:17.091404 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f3b29-18ce-4943-bc6e-c4d706de5f73 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:17.107884 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:17.109933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:17.110408 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:17.111634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] PATH_INFO: `/v3/projects/a11ecc186aa04a4ea292cc31eac3b4fe` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:17.130814 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11ecc186aa04a4ea292cc31eac3b4fe)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:17.133912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:17.295949 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11ecc186aa04a4ea292cc31eac3b4fe', 'request_id': 'req-fe3838ca-714e-4b09-9179-5493c16a2641'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.301663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11ecc186aa04a4ea292cc31eac3b4fe', 'request_id': 'req-fe3838ca-714e-4b09-9179-5493c16a2641'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.309431 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11ecc186aa04a4ea292cc31eac3b4fe', 'request_id': 'req-fe3838ca-714e-4b09-9179-5493c16a2641'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.395689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:17.395689 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:17.420353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292fb17-42af-4c51-b1aa-e3637ff97b29 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:17.427749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:17.434425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] The token cache is being invalidate because project a11ecc186aa04a4ea292cc31eac3b4fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:17.434425 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.436080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.455531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1155/2331] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:16 2022] DELETE /identity/v3/projects/a11ecc186aa04a4ea292cc31eac3b4fe => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:17.580572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:17.581326 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:17.581610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:17.581935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] PATH_INFO: `/v3/projects/1f14e4ba399d4349a0fe9ee6f7b894ed` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:17.583148 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f14e4ba399d4349a0fe9ee6f7b894ed)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:17.585228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:17.791683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f14e4ba399d4349a0fe9ee6f7b894ed', 'request_id': 'req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.798624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f14e4ba399d4349a0fe9ee6f7b894ed', 'request_id': 'req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.805745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f14e4ba399d4349a0fe9ee6f7b894ed', 'request_id': 'req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.944253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] The token cache is being invalidate because project 1f14e4ba399d4349a0fe9ee6f7b894ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:17.944253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.944253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:17.945649 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1177/2332] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:17 2022] DELETE /identity/v3/projects/1f14e4ba399d4349a0fe9ee6f7b894ed => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:18.180226 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:18.181604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:18.196028 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3838ca-714e-4b09-9179-5493c16a2641 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:18.270344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:18.271310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:18.271667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:18.272187 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] PATH_INFO: `/v3/users/996e99998ebc4e1ba42f0c14446c7706` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:18.273553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=996e99998ebc4e1ba42f0c14446c7706)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:18.275912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:18.480787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:18.502742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:18.507050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:18.556457 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '996e99998ebc4e1ba42f0c14446c7706'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:18.573430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '996e99998ebc4e1ba42f0c14446c7706'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:18.574657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '996e99998ebc4e1ba42f0c14446c7706'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:18.583607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '996e99998ebc4e1ba42f0c14446c7706'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:18.584632 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-cf31aef5-89e9-4f7f-8723-da1435f25757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '996e99998ebc4e1ba42f0c14446c7706'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:18.595935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc193ddb-3a40-4edb-99eb-58a00a2e0ab4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:18.601600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:18.602308 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:18.603290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:18.603574 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:18.606033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:18.716659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1156/2333] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:18 2022] DELETE /identity/v3/users/996e99998ebc4e1ba42f0c14446c7706 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:18.718729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:18.719018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:18.719320 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:18.756230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:18.757355 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:18.758489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:18.759087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:18.772256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1178/2334] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:18.864218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1157/2335] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:18.864403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:18.864796 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:18.928786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae538e36-ba57-4a1e-84ad-62455c02db36 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:19.007846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:19.007846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:19.007846 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:19.008253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:19.025040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:19.027301 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:19.041067 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1179/2336] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:19.909232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:19.911146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:19.931991 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b86444-c48b-468b-8ac4-f2f0acba2958 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:19.940614 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:19.942307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:19.943878 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:19.944162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] PATH_INFO: `/v3/projects/fb01e87771e043328bc21d1588988472` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:19.948136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb01e87771e043328bc21d1588988472)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:19.951271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:20.090109 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb01e87771e043328bc21d1588988472', 'request_id': 'req-164fc96a-e39c-43a6-8455-dfcfe371961b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.096160 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb01e87771e043328bc21d1588988472', 'request_id': 'req-164fc96a-e39c-43a6-8455-dfcfe371961b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.102485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb01e87771e043328bc21d1588988472', 'request_id': 'req-164fc96a-e39c-43a6-8455-dfcfe371961b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.235339 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] The token cache is being invalidate because project fb01e87771e043328bc21d1588988472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:20.235644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.236335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.238401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1158/2337] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:19 2022] DELETE /identity/v3/projects/fb01e87771e043328bc21d1588988472 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:20.242465 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:20.257124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0a622-1fad-4b35-9f98-de5f65207b5b service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:20.327186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:20.327866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:20.328231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:20.328504 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] PATH_INFO: `/v3/users/e276ed9fea284f3589032df8f877d583` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:20.329655 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e276ed9fea284f3589032df8f877d583)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:20.331677 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:20.692740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e276ed9fea284f3589032df8f877d583'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.693457 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e276ed9fea284f3589032df8f877d583'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.818458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e276ed9fea284f3589032df8f877d583'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.819058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:20.819231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e276ed9fea284f3589032df8f877d583'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.833609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e276ed9fea284f3589032df8f877d583'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:20.861891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1180/2338] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:20 2022] DELETE /identity/v3/users/e276ed9fea284f3589032df8f877d583 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:20.921048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:21.433763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:21.435462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:21.458161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fc96a-e39c-43a6-8455-dfcfe371961b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:21.628879 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:21.630278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:21.630851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:21.632118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] PATH_INFO: `/v3/projects/55b1ed0c7b8341da918970afb6783c61` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:21.637171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55b1ed0c7b8341da918970afb6783c61)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:21.641708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:21.894867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b1ed0c7b8341da918970afb6783c61', 'request_id': 'req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:21.902841 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b1ed0c7b8341da918970afb6783c61', 'request_id': 'req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:21.909901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b1ed0c7b8341da918970afb6783c61', 'request_id': 'req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.077158 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] The token cache is being invalidate because project 55b1ed0c7b8341da918970afb6783c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:22.077872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.079025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.081520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1159/2339] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:21 2022] DELETE /identity/v3/projects/55b1ed0c7b8341da918970afb6783c61 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:22.085242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:22.086981 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:22.101064 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e158f-7e0e-45c0-bb7e-84d1dd5fa850 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:22.158136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:22.158773 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:22.159051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:22.159323 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] PATH_INFO: `/v3/users/ac7f6d9946164f9ababbae19e8beb75e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:22.160514 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7f6d9946164f9ababbae19e8beb75e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:22.162604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:22.569393 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7f6d9946164f9ababbae19e8beb75e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.570157 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7f6d9946164f9ababbae19e8beb75e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.701228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7f6d9946164f9ababbae19e8beb75e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.702005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7f6d9946164f9ababbae19e8beb75e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.713265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6c42aeee-bd90-406e-80e6-55b417a86d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7f6d9946164f9ababbae19e8beb75e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:22.757344 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1181/2340] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:22 2022] DELETE /identity/v3/users/ac7f6d9946164f9ababbae19e8beb75e => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:23.315078 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:23.316624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:23.334853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf93bf0-156b-4c75-b0cd-50d3bbdafd07 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:23.418172 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:23.418851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:23.419660 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:23.420534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] PATH_INFO: `/v3/projects/6f53cb45bcb44873a53f257162a07fde` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:23.422248 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f53cb45bcb44873a53f257162a07fde)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:23.424275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:23.456622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:23.457089 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:23.457191 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:23.499724 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] MFA Rules not processed for user `a0312b8d61eb456c989ba3f57018e38c`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:23.501983 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:23.592192 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f53cb45bcb44873a53f257162a07fde', 'request_id': 'req-85aa52a4-0607-40e7-a72a-6c1b912518f6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:23.598608 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f53cb45bcb44873a53f257162a07fde', 'request_id': 'req-85aa52a4-0607-40e7-a72a-6c1b912518f6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:23.605908 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f53cb45bcb44873a53f257162a07fde', 'request_id': 'req-85aa52a4-0607-40e7-a72a-6c1b912518f6'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:23.632598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1182/2341] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:23.656372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:23.657964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:23.716857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] The token cache is being invalidate because project 6f53cb45bcb44873a53f257162a07fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:23.716857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:23.716857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-85aa52a4-0607-40e7-a72a-6c1b912518f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:23.716857 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1160/2342] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:23 2022] DELETE /identity/v3/projects/6f53cb45bcb44873a53f257162a07fde => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:23.719424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234bcf3-5afb-4774-a315-0af916ab4934 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:23.719719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234bcf3-5afb-4774-a315-0af916ab4934 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:23.720032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234bcf3-5afb-4774-a315-0af916ab4934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:23.749039 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1234bcf3-5afb-4774-a315-0af916ab4934 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:23.749851 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c871f-dbc7-43a7-873c-06ebb4129108 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:23.750430 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1234bcf3-5afb-4774-a315-0af916ab4934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:23.832950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:23.833565 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:23.833844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:23.834140 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:23.835951 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:23.842926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1161/2343] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:23.852644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29a82-aa3e-40ff-bff9-cd3148dcb985 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:23.852948 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29a82-aa3e-40ff-bff9-cd3148dcb985 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:23.853209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29a82-aa3e-40ff-bff9-cd3148dcb985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:23.875313 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-23e29a82-aa3e-40ff-bff9-cd3148dcb985 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:23.876647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-23e29a82-aa3e-40ff-bff9-cd3148dcb985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:23.886164 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1162/2344] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:23.890664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29a82-aa3e-40ff-bff9-cd3148dcb985 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:23.901498 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29a82-aa3e-40ff-bff9-cd3148dcb985 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:23.906859 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:23.907475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:23.907755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:23.908061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:23.908400 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:23.908713 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:13:23.909019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:23.910225 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:23.910694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:23.922479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1183/2345] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:13:23.925232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1163/2346] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:13:23.930577 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:23.942915 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0ae97-d21d-48bc-8d7a-3d873afcd354 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:23.948319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:23.948755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:23.949043 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:23.949437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:23.950114 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:23.952925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:24.014414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:24.085840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1184/2347] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:23 2022] POST /identity/v3/projects => generated 401 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:24.089235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:24.100282 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621eab-b59c-4301-a629-8d64e94ba7b6 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:24.107650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:24.177634 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:24.178522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:24.178722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:24.179074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:24.179732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:24.181741 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:24.372434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1164/2348] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:24 2022] POST /identity/v3/users => generated 301 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:24.376809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:24.404486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e959a-d1e7-4f7d-91f8-9aa905d25b6c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:24.413118 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:24.413863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:24.414228 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:24.414564 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:24.415452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:24.417380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:24.431830 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1185/2349] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:24.436061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:24.447551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d9f74-df9a-47b8-be44-f7d0102633fd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:24.453854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6d1b6-522a-4931-90db-73d669d1f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:24.454487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6d1b6-522a-4931-90db-73d669d1f45b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:24.454762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6d1b6-522a-4931-90db-73d669d1f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:24.455059 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6d1b6-522a-4931-90db-73d669d1f45b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:24.455752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6d1b6-522a-4931-90db-73d669d1f45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:24.457386 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6d1b6-522a-4931-90db-73d669d1f45b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:24.470497 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1165/2350] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:13:24.479246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:24.490669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a7ac7-e0e4-4c57-925a-bb2440bffaf9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:24.495458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:24.496171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:24.496489 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:24.496774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] PATH_INFO: `/v3/projects/d00c18fdeba74b15aafdb8cd64e9606e/users/542896dc4a59489c9682067245e4d08d/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:24.527027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00c18fdeba74b15aafdb8cd64e9606e, user_id=542896dc4a59489c9682067245e4d08d, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:24.534733 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:24.663593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1a42e-6c23-475f-8e9c-4282e5ce424c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:24.664823 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1a42e-6c23-475f-8e9c-4282e5ce424c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:24.666914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1a42e-6c23-475f-8e9c-4282e5ce424c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:24.679395 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1186/2351] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:24 2022] PUT /identity/v3/projects/d00c18fdeba74b15aafdb8cd64e9606e/users/542896dc4a59489c9682067245e4d08d/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:24.720975 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c6f1a42e-6c23-475f-8e9c-4282e5ce424c None None] MFA Rules not processed for user `542896dc4a59489c9682067245e4d08d`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:24.722221 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c6f1a42e-6c23-475f-8e9c-4282e5ce424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:24.795918 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1166/2352] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:24 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:24.799943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:24.904947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a8af-68b6-4f97-9179-45c07c18fbcd admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:24.910466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:24.911231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:24.911399 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:24.912037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:24.912977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:24.915219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:25.051044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:25.060078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1187/2353] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:24 2022] POST /identity/v3/projects => generated 399 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:25.063836 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1a42e-6c23-475f-8e9c-4282e5ce424c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:25.076528 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1a42e-6c23-475f-8e9c-4282e5ce424c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:25.143060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:25.143804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:25.144184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:25.144538 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:25.145241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:25.147178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:25.314886 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1167/2354] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:25 2022] POST /identity/v3/users => generated 306 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:25.319184 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:25.332159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76db5f-d81e-489a-a294-98b5d7cfe605 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:25.337824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:25.338923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:25.338923 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:25.339119 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:25.339643 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:25.341602 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:25.353631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1188/2355] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:13:25.358072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:25.368125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0479e0-cabd-40e3-814c-64aa9c3e041a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:25.373751 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:25.374376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:25.374656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:25.374933 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] PATH_INFO: `/v3/projects/2ee35711775b4b99a7ec301383f10087/users/47b34f08802b42598cfa1432f6917e45/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:25.390901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee35711775b4b99a7ec301383f10087, user_id=47b34f08802b42598cfa1432f6917e45, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:25.393610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:25.510006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1168/2356] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:25 2022] PUT /identity/v3/projects/2ee35711775b4b99a7ec301383f10087/users/47b34f08802b42598cfa1432f6917e45/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:25.515005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:25.525091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b834-cadc-4aca-b81f-4221a45944cd admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:25.581575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cf71c-cea9-49ac-a97c-0cf0ae571b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:25.582194 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cf71c-cea9-49ac-a97c-0cf0ae571b80 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:25.582492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cf71c-cea9-49ac-a97c-0cf0ae571b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:25.582755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cf71c-cea9-49ac-a97c-0cf0ae571b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:25.583372 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cf71c-cea9-49ac-a97c-0cf0ae571b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:25.584986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cf71c-cea9-49ac-a97c-0cf0ae571b80 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:25.594953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1189/2357] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:25.599286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:25.610161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0881cb5-031e-4b64-9fed-a0bb67aaaa6e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:25.614671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:25.615312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:25.615715 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:25.615889 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] PATH_INFO: `/v3/projects/2ee35711775b4b99a7ec301383f10087/users/47b34f08802b42598cfa1432f6917e45/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:25.617777 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee35711775b4b99a7ec301383f10087, user_id=47b34f08802b42598cfa1432f6917e45, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:25.620435 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:25.630987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1169/2358] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:25 2022] PUT /identity/v3/projects/2ee35711775b4b99a7ec301383f10087/users/47b34f08802b42598cfa1432f6917e45/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:13:25.641468 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d45b5-4771-4e3a-83f8-2219414ba951 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:25.641754 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d45b5-4771-4e3a-83f8-2219414ba951 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:25.642029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d45b5-4771-4e3a-83f8-2219414ba951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:25.669934 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9c4d45b5-4771-4e3a-83f8-2219414ba951 None None] MFA Rules not processed for user `47b34f08802b42598cfa1432f6917e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:25.671249 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9c4d45b5-4771-4e3a-83f8-2219414ba951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:25.745052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1190/2359] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:25.759500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:25.761506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:25.832934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d14947-3e67-4b24-b16a-cb96bfdb710c admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:25.838175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:25.838955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:25.839247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:25.839518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:25.849721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:25.851144 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:25.861649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1170/2360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:26.139479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d45b5-4771-4e3a-83f8-2219414ba951 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:26.149522 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d45b5-4771-4e3a-83f8-2219414ba951 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:26.153328 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:26.154936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:26.158958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:26.158958 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:26.218471 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:26.220112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:26.229245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1191/2361] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:26 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:26.771296 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:28.712066 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:30.694231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:30.763180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:31.562794 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:32.550347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:32.551743 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:32.621928 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294602a-2fab-44db-a4e5-fa6b71dc6756 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:32.627837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920a6dd-b2db-4866-ac16-043143a3f29f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c154983aff14d9895fb4b542db61e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:32.628488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a6dd-b2db-4866-ac16-043143a3f29f service placement] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:32.628767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a6dd-b2db-4866-ac16-043143a3f29f service placement] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:32.629038 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a6dd-b2db-4866-ac16-043143a3f29f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:32.630840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a920a6dd-b2db-4866-ac16-043143a3f29f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:32.643649 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920a6dd-b2db-4866-ac16-043143a3f29f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:32.645162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920a6dd-b2db-4866-ac16-043143a3f29f service placement] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:32.656866 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1171/2362] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:35.323626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:35.355987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:35.388146 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65513d-93fc-488a-8ce3-4fbb6d000cd6 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:35.393234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:35.393877 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:35.394380 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:35.394828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] PATH_INFO: `/v3/users/542896dc4a59489c9682067245e4d08d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:35.396281 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542896dc4a59489c9682067245e4d08d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:35.398233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:35.643746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542896dc4a59489c9682067245e4d08d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:35.644423 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542896dc4a59489c9682067245e4d08d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:35.743601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542896dc4a59489c9682067245e4d08d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:35.744412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542896dc4a59489c9682067245e4d08d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:35.753159 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542896dc4a59489c9682067245e4d08d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:35.772221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1192/2363] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:35 2022] DELETE /identity/v3/users/542896dc4a59489c9682067245e4d08d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:35.776787 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:35.777241 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:35.777440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:35.802303 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:35.803494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:35.865092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1172/2364] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:35.875755 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:35.877524 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:35.936505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d29ee-b5b0-4ce6-a601-26a0d93bc5d8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:35.940737 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:35.941350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:35.946828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:35.946828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:35.951021 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:35.953186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:35.962961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1193/2365] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:36.435988 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:36.438136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:36.449707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0a7eb-2ed4-473e-a27f-8c15c1c98345 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:36.455369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:36.455369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:36.455369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:36.455723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] PATH_INFO: `/v3/projects/d00c18fdeba74b15aafdb8cd64e9606e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:36.456640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00c18fdeba74b15aafdb8cd64e9606e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:36.458558 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:36.590462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:36.591876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00c18fdeba74b15aafdb8cd64e9606e', 'request_id': 'req-424cd6b1-a04c-403d-bd34-a08c88ea5e38'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:36.598231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00c18fdeba74b15aafdb8cd64e9606e', 'request_id': 'req-424cd6b1-a04c-403d-bd34-a08c88ea5e38'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:36.604487 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00c18fdeba74b15aafdb8cd64e9606e', 'request_id': 'req-424cd6b1-a04c-403d-bd34-a08c88ea5e38'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:36.751132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] The token cache is being invalidate because project d00c18fdeba74b15aafdb8cd64e9606e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:36.751361 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:36.752051 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:36.753640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1173/2366] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:36 2022] DELETE /identity/v3/projects/d00c18fdeba74b15aafdb8cd64e9606e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:36.757711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:36.767917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c0d4-c999-4766-8b96-9c4ba1b9a671 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:36.816485 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:36.826220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:36.826862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:36.827162 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:36.827996 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] PATH_INFO: `/v3/users/47b34f08802b42598cfa1432f6917e45` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:36.829233 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b34f08802b42598cfa1432f6917e45)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:36.831214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:37.052554 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b34f08802b42598cfa1432f6917e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.053200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b34f08802b42598cfa1432f6917e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.154669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b34f08802b42598cfa1432f6917e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.157973 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b34f08802b42598cfa1432f6917e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.163037 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-595caaa4-bf5b-407f-9663-cc6ae4c6601e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b34f08802b42598cfa1432f6917e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.181573 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1194/2367] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:36 2022] DELETE /identity/v3/users/47b34f08802b42598cfa1432f6917e45 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:37.627152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:37.628791 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:37.657421 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cd6b1-a04c-403d-bd34-a08c88ea5e38 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:37.714942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:37.715619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:37.715938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:37.716216 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] PATH_INFO: `/v3/projects/2ee35711775b4b99a7ec301383f10087` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:37.717647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee35711775b4b99a7ec301383f10087)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:37.720444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:37.846494 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee35711775b4b99a7ec301383f10087', 'request_id': 'req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.852873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee35711775b4b99a7ec301383f10087', 'request_id': 'req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.853786 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82144742-dd28-4562-a3f1-6bd127cd818c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:37.854291 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82144742-dd28-4562-a3f1-6bd127cd818c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:37.854388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82144742-dd28-4562-a3f1-6bd127cd818c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:37.859996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee35711775b4b99a7ec301383f10087', 'request_id': 'req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.874965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-82144742-dd28-4562-a3f1-6bd127cd818c None None] MFA Rules not processed for user `914b60bb9bfa471880700762562e24f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:37.876161 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-82144742-dd28-4562-a3f1-6bd127cd818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:37.931833 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1195/2368] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:37.972628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] The token cache is being invalidate because project 2ee35711775b4b99a7ec301383f10087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:37.972913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.973531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a0fbc4e4-4c26-4e3d-aaf4-f0122a14dd29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:37.975069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1174/2369] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:37 2022] DELETE /identity/v3/projects/2ee35711775b4b99a7ec301383f10087 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:37.980077 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e72c5-b5f4-42c6-bb6c-2666a7b36d90 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:37.980365 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e72c5-b5f4-42c6-bb6c-2666a7b36d90 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:37.980640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e72c5-b5f4-42c6-bb6c-2666a7b36d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.007739 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fc0e72c5-b5f4-42c6-bb6c-2666a7b36d90 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:38.009360 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fc0e72c5-b5f4-42c6-bb6c-2666a7b36d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:38.075929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1196/2370] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:38.085288 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c0def-6264-4a99-b6d0-9f191ac8719c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.085604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c0def-6264-4a99-b6d0-9f191ac8719c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.085881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c0def-6264-4a99-b6d0-9f191ac8719c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.106994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-c25c0def-6264-4a99-b6d0-9f191ac8719c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:38.108306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-c25c0def-6264-4a99-b6d0-9f191ac8719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:38.116946 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1175/2371] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:38.121415 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e72c5-b5f4-42c6-bb6c-2666a7b36d90 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:38.131748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e72c5-b5f4-42c6-bb6c-2666a7b36d90 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:38.137099 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:38.137709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.137986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.138278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.138903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:13:38.139202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:38.141742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:38.153905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1197/2372] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:13:38.159235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c0def-6264-4a99-b6d0-9f191ac8719c None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:38.169198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c0def-6264-4a99-b6d0-9f191ac8719c None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:38.175111 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:38.175716 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.176033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.176310 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.176945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:38.178825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:38.310927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1176/2373] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:38 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:38.315127 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:38.320253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:38.325500 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3769d-beb8-4675-95ef-6bc1125a9a29 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:38.382109 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:38.382661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.382937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.383205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.384135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:38.386121 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:38.533156 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1198/2374] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:38 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:38.537292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:38.547995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924736-43ab-4d7b-b31e-7311b51df1a9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:38.552502 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:38.553154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.553427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.553692 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.554318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:38.555941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:38.566228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1177/2375] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:38.570198 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:38.580699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436e340-fc50-4895-bafe-7243f5c048bc admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:38.585356 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cc71b-24f4-4e8e-b462-782af8da94c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:38.585945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc71b-24f4-4e8e-b462-782af8da94c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.586223 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc71b-24f4-4e8e-b462-782af8da94c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.586491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cc71b-24f4-4e8e-b462-782af8da94c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.587189 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cc71b-24f4-4e8e-b462-782af8da94c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:38.588818 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cc71b-24f4-4e8e-b462-782af8da94c8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:38.599982 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1199/2376] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:13:38.603998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:38.614571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b71bd-bd35-41d5-bac6-6565adb86731 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:38.619205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:38.619881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.620175 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.620448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] PATH_INFO: `/v3/projects/ccc50abb186448d08d1cc13982bd1a06/users/9a373fc04381459fa1745e46595a0d38/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.637724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc50abb186448d08d1cc13982bd1a06, user_id=9a373fc04381459fa1745e46595a0d38, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:38.641620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:38.756289 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1178/2377] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:38 2022] PUT /identity/v3/projects/ccc50abb186448d08d1cc13982bd1a06/users/9a373fc04381459fa1745e46595a0d38/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:38.767463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305cef4-564b-4253-8692-2ebfa5be1df1 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:38.767746 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305cef4-564b-4253-8692-2ebfa5be1df1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:38.768052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305cef4-564b-4253-8692-2ebfa5be1df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:38.797150 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7305cef4-564b-4253-8692-2ebfa5be1df1 None None] MFA Rules not processed for user `9a373fc04381459fa1745e46595a0d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:38.798358 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7305cef4-564b-4253-8692-2ebfa5be1df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:38.870172 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1200/2378] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:41.946161 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:41.948168 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:41.953271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:42.014009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5e2c7-deb4-497c-8457-a54644578926 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:42.018817 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:42.019449 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:42.019752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:42.020060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] PATH_INFO: `/v3/users/ce2972d552564baca6fa686b84ff45ba` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:42.021193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2972d552564baca6fa686b84ff45ba)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:42.023140 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:42.272695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2972d552564baca6fa686b84ff45ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.286103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2972d552564baca6fa686b84ff45ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.286828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2972d552564baca6fa686b84ff45ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.294532 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2972d552564baca6fa686b84ff45ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.295198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2972d552564baca6fa686b84ff45ba'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.395666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1179/2379] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:41 2022] DELETE /identity/v3/users/ce2972d552564baca6fa686b84ff45ba => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:42.655901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305cef4-564b-4253-8692-2ebfa5be1df1 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:42.668004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305cef4-564b-4253-8692-2ebfa5be1df1 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:42.725784 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:42.726419 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:42.726745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:42.727022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] PATH_INFO: `/v3/projects/0ac7469cc0804ddb9702159dddc67b39` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:42.728187 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac7469cc0804ddb9702159dddc67b39)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:42.730110 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:42.741613 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:42.877644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac7469cc0804ddb9702159dddc67b39', 'request_id': 'req-aba16470-7786-4c38-bc25-b1032dd1bda9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.883534 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac7469cc0804ddb9702159dddc67b39', 'request_id': 'req-aba16470-7786-4c38-bc25-b1032dd1bda9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.890863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac7469cc0804ddb9702159dddc67b39', 'request_id': 'req-aba16470-7786-4c38-bc25-b1032dd1bda9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.983214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] The token cache is being invalidate because project 0ac7469cc0804ddb9702159dddc67b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:42.983507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.984673 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:42.986342 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1201/2380] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:42 2022] DELETE /identity/v3/projects/0ac7469cc0804ddb9702159dddc67b39 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:42.990343 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:42.995468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:43.005940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3602e24-354e-42e8-baef-172b7605b7a3 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:43.063827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:43.064450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:43.064727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:43.065000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] PATH_INFO: `/v3/users/e48eb00bef4f49d9b2cdc2ca12dba22c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:43.066120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48eb00bef4f49d9b2cdc2ca12dba22c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:43.068132 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:43.090446 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:43.332519 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48eb00bef4f49d9b2cdc2ca12dba22c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:43.346514 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48eb00bef4f49d9b2cdc2ca12dba22c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:43.347209 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48eb00bef4f49d9b2cdc2ca12dba22c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:43.354659 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48eb00bef4f49d9b2cdc2ca12dba22c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:43.355285 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6b1bf05b-2261-4072-bd3a-25157e475f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48eb00bef4f49d9b2cdc2ca12dba22c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:43.461405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1180/2381] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:42 2022] DELETE /identity/v3/users/e48eb00bef4f49d9b2cdc2ca12dba22c => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:43.710777 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:43.712315 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:43.727479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16470-7786-4c38-bc25-b1032dd1bda9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:43.788385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:43.789041 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:43.789321 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:43.789591 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] PATH_INFO: `/v3/projects/39f7d996ceea495995116b3ccceb0c40` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:43.790741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f7d996ceea495995116b3ccceb0c40)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:43.792727 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:43.939163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f7d996ceea495995116b3ccceb0c40', 'request_id': 'req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:43.944946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f7d996ceea495995116b3ccceb0c40', 'request_id': 'req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:43.951230 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f7d996ceea495995116b3ccceb0c40', 'request_id': 'req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:44.022563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] The token cache is being invalidate because project 39f7d996ceea495995116b3ccceb0c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:44.022853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:44.023483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-440a0b5f-5f0e-4c6c-a572-a5d2c40a00f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:44.025124 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1202/2382] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:43 2022] DELETE /identity/v3/projects/39f7d996ceea495995116b3ccceb0c40 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:44.029917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db32d-e8d9-4b90-a307-03027947c9e0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.030396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db32d-e8d9-4b90-a307-03027947c9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.030506 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db32d-e8d9-4b90-a307-03027947c9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.052287 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-321db32d-e8d9-4b90-a307-03027947c9e0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:44.053512 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-321db32d-e8d9-4b90-a307-03027947c9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:44.115376 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1181/2383] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:44.123289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173f9d-fb39-4b17-85df-1ba26f0892cb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.123572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173f9d-fb39-4b17-85df-1ba26f0892cb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.123887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173f9d-fb39-4b17-85df-1ba26f0892cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.143626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-44173f9d-fb39-4b17-85df-1ba26f0892cb None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:44.144854 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-44173f9d-fb39-4b17-85df-1ba26f0892cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:44.152568 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1203/2384] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:44.157472 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321db32d-e8d9-4b90-a307-03027947c9e0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:44.167072 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321db32d-e8d9-4b90-a307-03027947c9e0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:44.171747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:44.172952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.173274 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.173533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.174179 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:13:44.174480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:44.176116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:44.187700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1182/2385] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:13:44.191952 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44173f9d-fb39-4b17-85df-1ba26f0892cb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:44.201049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44173f9d-fb39-4b17-85df-1ba26f0892cb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:44.205647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:44.206234 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.206509 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.206776 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.207484 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:44.209396 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:44.334442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1204/2386] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:44 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:44.335582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:44.346401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2079dcb-bf42-4cad-9d97-73bde26cce4d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:44.418257 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:44.418976 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.419553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.419689 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.420723 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:44.423631 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:44.535871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1183/2387] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:44 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:44.539935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:44.545866 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:44.551412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19bcc6-d3e1-40d0-87c9-c1a0f03f4333 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:44.555943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:44.556559 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.556840 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.557113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.557748 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:44.559450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:44.569379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1205/2388] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:13:44.573272 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:44.583462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe5b25-d260-4b65-9ddc-43d77d4a2184 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:44.588019 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f4817-be0e-4955-86c6-19bb0bece783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:44.588594 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4817-be0e-4955-86c6-19bb0bece783 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.588870 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4817-be0e-4955-86c6-19bb0bece783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.589138 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4817-be0e-4955-86c6-19bb0bece783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.589767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f4817-be0e-4955-86c6-19bb0bece783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:44.591394 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f4817-be0e-4955-86c6-19bb0bece783 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:44.602825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1184/2389] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:13:44.606730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:44.616917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d9921-4a45-4e2d-a217-85afd4556d1c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:44.621450 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:44.622129 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.622647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.622918 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] PATH_INFO: `/v3/projects/925554bf4d9c46c0bf80fb786683024c/users/64e8623a804149f9a80feab4964e4396/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.639306 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925554bf4d9c46c0bf80fb786683024c, user_id=64e8623a804149f9a80feab4964e4396, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:44.643034 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:44.739135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1206/2390] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:44 2022] PUT /identity/v3/projects/925554bf4d9c46c0bf80fb786683024c/users/64e8623a804149f9a80feab4964e4396/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:44.749269 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9009e2-12c1-4572-86b9-db9fc3c0dbe4 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.749552 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9009e2-12c1-4572-86b9-db9fc3c0dbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.750305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9009e2-12c1-4572-86b9-db9fc3c0dbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.764293 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:44.779978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-ed9009e2-12c1-4572-86b9-db9fc3c0dbe4 None None] MFA Rules not processed for user `64e8623a804149f9a80feab4964e4396`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:44.781201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-ed9009e2-12c1-4572-86b9-db9fc3c0dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:44.845250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1185/2391] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:44.857214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:44.860532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:44.919219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c19439-f9d9-4542-8926-c58a089c99d9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:44.923247 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550ede1-af1a-49c9-bb28-3c268712eee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:44.923891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ede1-af1a-49c9-bb28-3c268712eee1 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:44.924170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ede1-af1a-49c9-bb28-3c268712eee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:44.924438 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550ede1-af1a-49c9-bb28-3c268712eee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:44.932765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550ede1-af1a-49c9-bb28-3c268712eee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:44.934179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550ede1-af1a-49c9-bb28-3c268712eee1 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:44.943731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1207/2392] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:44.967167 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9009e2-12c1-4572-86b9-db9fc3c0dbe4 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:45.030955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9009e2-12c1-4572-86b9-db9fc3c0dbe4 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:45.035816 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:45.036452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:45.036733 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:45.037007 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] PATH_INFO: `/v3/users/64e8623a804149f9a80feab4964e4396` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:45.038152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e8623a804149f9a80feab4964e4396)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:45.040212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:45.226811 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e8623a804149f9a80feab4964e4396'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:45.239674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e8623a804149f9a80feab4964e4396'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:45.240391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e8623a804149f9a80feab4964e4396'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:45.247407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e8623a804149f9a80feab4964e4396'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:45.248092 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e8623a804149f9a80feab4964e4396'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:45.424576 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1186/2393] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:13:44 2022] DELETE /identity/v3/users/64e8623a804149f9a80feab4964e4396 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:45.424720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:45.424720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:45.424720 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:45.455989 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:45.457430 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:45.521747 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1208/2394] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:13:45.532369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:45.533887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:45.556256 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:45.600888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260289e3-e2d7-4371-81b3-f35b15ffc8c9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:45.605275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef30846-4bcc-483f-9f02-0a97aec5c17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:45.606020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30846-4bcc-483f-9f02-0a97aec5c17b service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:45.606372 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30846-4bcc-483f-9f02-0a97aec5c17b service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:45.606688 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30846-4bcc-483f-9f02-0a97aec5c17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:45.617456 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef30846-4bcc-483f-9f02-0a97aec5c17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:45.619004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef30846-4bcc-483f-9f02-0a97aec5c17b service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:45.629095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1187/2395] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:46.221749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:46.221749 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:46.241265 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c033b1-1f83-4925-8934-338482c4b1a8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:46.253245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:46.253245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:46.253245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:46.253245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] PATH_INFO: `/v3/projects/925554bf4d9c46c0bf80fb786683024c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:46.253804 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=925554bf4d9c46c0bf80fb786683024c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:46.256639 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:46.387458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925554bf4d9c46c0bf80fb786683024c', 'request_id': 'req-2b3719bd-a285-402c-9ff0-444c32b5c4d0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:46.393537 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925554bf4d9c46c0bf80fb786683024c', 'request_id': 'req-2b3719bd-a285-402c-9ff0-444c32b5c4d0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:46.399735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925554bf4d9c46c0bf80fb786683024c', 'request_id': 'req-2b3719bd-a285-402c-9ff0-444c32b5c4d0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:46.402273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:46.469546 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] The token cache is being invalidate because project 925554bf4d9c46c0bf80fb786683024c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:13:46.469865 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:46.470593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b3719bd-a285-402c-9ff0-444c32b5c4d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:13:46.472143 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1209/2396] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:13:46 2022] DELETE /identity/v3/projects/925554bf4d9c46c0bf80fb786683024c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:46.477088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d8295-80d2-44a5-973d-ef1333c64325 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:46.477604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d8295-80d2-44a5-973d-ef1333c64325 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:46.477797 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d8295-80d2-44a5-973d-ef1333c64325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:46.498128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-698d8295-80d2-44a5-973d-ef1333c64325 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:46.499356 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-698d8295-80d2-44a5-973d-ef1333c64325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:46.564341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1188/2397] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:46.573056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71c13f-a9ba-49ca-ac32-514bc6eedca4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:46.573343 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71c13f-a9ba-49ca-ac32-514bc6eedca4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:46.573622 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71c13f-a9ba-49ca-ac32-514bc6eedca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:46.594744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5d71c13f-a9ba-49ca-ac32-514bc6eedca4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:46.596063 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5d71c13f-a9ba-49ca-ac32-514bc6eedca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:46.603780 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1210/2398] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:46.607938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d8295-80d2-44a5-973d-ef1333c64325 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:46.618598 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d8295-80d2-44a5-973d-ef1333c64325 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:46.623278 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:46.623912 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:46.624189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:46.624460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] PATH_INFO: `/v3/domains` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:46.625727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81608) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:13:46.625727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:46.627605 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:46.639204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1189/2399] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:13:46.643929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71c13f-a9ba-49ca-ac32-514bc6eedca4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:46.653017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71c13f-a9ba-49ca-ac32-514bc6eedca4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:46.657892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:46.658529 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:46.658803 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:46.659065 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:46.659683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:46.661582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:46.764418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1211/2400] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:46 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:13:46.768510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:46.778436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019732d-64f3-4461-81c3-75f221465aed admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:46.835357 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:46.835983 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:46.836262 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:46.836533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:46.837150 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:46.839099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:46.946360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1190/2401] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:46 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:46.950962 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:46.962604 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224e25c-82a1-465f-bde2-1cb2d2c72638 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:46.968437 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:46.969941 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:46.970401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:46.970844 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:46.971700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:46.973411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:46.985730 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1212/2402] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:46.989826 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:47.000314 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbde3ee-6b00-462c-97d6-04b5018822ba admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:47.006141 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf508b-05b8-46c2-b951-1796572af36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:47.006872 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf508b-05b8-46c2-b951-1796572af36e admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:47.007152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf508b-05b8-46c2-b951-1796572af36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:47.007424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf508b-05b8-46c2-b951-1796572af36e admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:47.008205 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf508b-05b8-46c2-b951-1796572af36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:47.010383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf508b-05b8-46c2-b951-1796572af36e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:47.025858 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1191/2403] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:47.036978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:47.051175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce97ab-f384-48c7-b1a7-bcc3a8959b22 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:47.056280 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:47.056970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:47.057251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:47.057523 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] PATH_INFO: `/v3/projects/4a9ff4fa4ae34163b8e9ea42fae1d723/users/14327353a11e4027bde6c57b923eb185/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:47.073389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9ff4fa4ae34163b8e9ea42fae1d723, user_id=14327353a11e4027bde6c57b923eb185, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:47.076071 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:47.146394 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1213/2404] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:47 2022] PUT /identity/v3/projects/4a9ff4fa4ae34163b8e9ea42fae1d723/users/14327353a11e4027bde6c57b923eb185/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:47.151663 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec5ca9-17d1-491d-8e5b-4e5016a43e8c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:47.152004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec5ca9-17d1-491d-8e5b-4e5016a43e8c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:47.152276 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec5ca9-17d1-491d-8e5b-4e5016a43e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:47.173118 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-35ec5ca9-17d1-491d-8e5b-4e5016a43e8c None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:47.174370 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-35ec5ca9-17d1-491d-8e5b-4e5016a43e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:47.193368 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:47.233839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1192/2405] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:47.245277 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:47.247097 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:47.305314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598872-c513-4d11-b123-6392158b5f60 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:47.308901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:47.309505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:47.309782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:47.310054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:47.319245 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:47.320701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:47.330765 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1214/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:52.979914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c769f-12c1-45e1-b281-952961ece397 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:52.979914 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c769f-12c1-45e1-b281-952961ece397 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:52.980576 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c769f-12c1-45e1-b281-952961ece397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:53.028929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-691c769f-12c1-45e1-b281-952961ece397 None None] MFA Rules not processed for user `14327353a11e4027bde6c57b923eb185`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:53.030170 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-691c769f-12c1-45e1-b281-952961ece397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:53.113230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1193/2407] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:13:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:53.117815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:53.131138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b59a21-500c-44cc-9615-fd84ea6a10f4 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:53.136224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:53.136862 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:53.137601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:53.137601 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:53.138054 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:53.140019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:53.248472 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1215/2408] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:13:53 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:53.253250 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c769f-12c1-45e1-b281-952961ece397 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:53.264959 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c769f-12c1-45e1-b281-952961ece397 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:53.331220 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:53.331903 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:53.332207 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:53.332485 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:53.333174 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:53.335146 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:53.455193 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1194/2409] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:13:53 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:13:53.459149 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:53.469584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a69f87-f7be-40cb-8b8b-424de96cbd31 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:53.474647 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:53.475243 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:53.475516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:53.475886 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:53.477244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:53.478552 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:53.492248 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1216/2410] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:53.496429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:53.509215 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c083f2-3e9f-45fa-9105-9ee7b8be9e1f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:53.514646 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:53.515571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:53.516419 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:53.516693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] PATH_INFO: `/v3/projects/23ac2b9dc837463fbb94cac1140ad915/users/39493a0141f54214801f3291b20f8dc3/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:53.534166 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ac2b9dc837463fbb94cac1140ad915, user_id=39493a0141f54214801f3291b20f8dc3, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:53.536906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:53.615352 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1195/2411] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:53 2022] PUT /identity/v3/projects/23ac2b9dc837463fbb94cac1140ad915/users/39493a0141f54214801f3291b20f8dc3/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:13:53.619411 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:53.630216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edb22b-032e-45db-b3ba-5e859babc30b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:53.695180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913a9e1-a7e0-457d-9955-d75c27011d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:53.695815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a9e1-a7e0-457d-9955-d75c27011d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:53.696100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a9e1-a7e0-457d-9955-d75c27011d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:53.696373 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a9e1-a7e0-457d-9955-d75c27011d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:53.697015 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913a9e1-a7e0-457d-9955-d75c27011d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:53.698607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913a9e1-a7e0-457d-9955-d75c27011d87 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:53.700009 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:53.714175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1217/2412] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:13:53.715401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:53.732533 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170693a-519e-4766-9a3a-dd71f50b26e0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:53.742468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:53.743152 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:53.743428 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:53.743700 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] PATH_INFO: `/v3/projects/23ac2b9dc837463fbb94cac1140ad915/users/39493a0141f54214801f3291b20f8dc3/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:53.745680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ac2b9dc837463fbb94cac1140ad915, user_id=39493a0141f54214801f3291b20f8dc3, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:53.748500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:53.761184 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1196/2413] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:13:53 2022] PUT /identity/v3/projects/23ac2b9dc837463fbb94cac1140ad915/users/39493a0141f54214801f3291b20f8dc3/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:00.127932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56f080-6554-499e-aa49-10d068662e6b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:00.128463 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56f080-6554-499e-aa49-10d068662e6b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:00.128576 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56f080-6554-499e-aa49-10d068662e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:00.186499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ce56f080-6554-499e-aa49-10d068662e6b None None] MFA Rules not processed for user `39493a0141f54214801f3291b20f8dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:00.186499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ce56f080-6554-499e-aa49-10d068662e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:00.268176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1218/2414] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:14:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:03.756828 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:10.264513 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=36 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:11.699378 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:11.701104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:11.760783 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f9e4-464d-4d37-8fec-73c5fe46a296 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:11.764359 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed8ccc-3440-4953-a813-dfa4263c38ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9627e6a9e86e43b694f723bd4109460e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:11.764954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed8ccc-3440-4953-a813-dfa4263c38ec service glance] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:11.765228 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed8ccc-3440-4953-a813-dfa4263c38ec service glance] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:11.765492 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed8ccc-3440-4953-a813-dfa4263c38ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:11.767375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-87ed8ccc-3440-4953-a813-dfa4263c38ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:11.825624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed8ccc-3440-4953-a813-dfa4263c38ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:11.827064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed8ccc-3440-4953-a813-dfa4263c38ec service glance] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:11.837739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1197/2415] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:14:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:14:12.137074 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f9a-0c8c-42af-8639-07e8ed24da90 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:12.137587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f9a-0c8c-42af-8639-07e8ed24da90 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:12.137690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f9a-0c8c-42af-8639-07e8ed24da90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-99100f9a-0c8c-42af-8639-07e8ed24da90 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:14:12.150017 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:14:12.151057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:14:12.152289 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1219/2416] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:14:12 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 15 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:14:12.187066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cdf6c-3d47-42c1-8579-b4418a9600cb None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:12.187412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cdf6c-3d47-42c1-8579-b4418a9600cb None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:12.187759 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cdf6c-3d47-42c1-8579-b4418a9600cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:12.191115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100f9a-0c8c-42af-8639-07e8ed24da90 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:12.193802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-99100f9a-0c8c-42af-8639-07e8ed24da90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-a02cdf6c-3d47-42c1-8579-b4418a9600cb None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:14:12.197800 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:14:12.198652 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 11:14:12.199520 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1198/2417] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:14:12 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 13 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:14:12.280130 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100f9a-0c8c-42af-8639-07e8ed24da90 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:12.285615 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:12.288802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:12.288802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:12.288802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:12.347221 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:12.348937 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:12.358895 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1220/2418] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:14:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:14:15.389337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2d0c-dec5-4c96-8be1-e8fe86d2fcac None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:15.389863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2d0c-dec5-4c96-8be1-e8fe86d2fcac None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:15.389863 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e2d0c-dec5-4c96-8be1-e8fe86d2fcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:15.409687 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-b64e2d0c-dec5-4c96-8be1-e8fe86d2fcac None None] MFA Rules not processed for user `14327353a11e4027bde6c57b923eb185`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:15.410892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-b64e2d0c-dec5-4c96-8be1-e8fe86d2fcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:15.417941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1199/2419] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:14:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:22.355133 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=56 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:25.404211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=51 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:40.313769 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:40.315364 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:40.385204 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de21f04-5a20-41e4-91d2-b0b04cfa9e99 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:40.390620 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:40.391409 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:40.391741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:40.392079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] PATH_INFO: `/v3/users/914b60bb9bfa471880700762562e24f1` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:40.393309 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914b60bb9bfa471880700762562e24f1)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:40.395333 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:41.053033 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914b60bb9bfa471880700762562e24f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.054319 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914b60bb9bfa471880700762562e24f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.107062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914b60bb9bfa471880700762562e24f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.108921 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914b60bb9bfa471880700762562e24f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.122098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-90bfcf89-cb72-436b-9749-81d31abe2cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914b60bb9bfa471880700762562e24f1'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.150977 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1221/2420] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:14:40 2022] DELETE /identity/v3/users/914b60bb9bfa471880700762562e24f1 => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:41.428579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2d0c-dec5-4c96-8be1-e8fe86d2fcac None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:41.446597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e2d0c-dec5-4c96-8be1-e8fe86d2fcac None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:41.518916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:41.519583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:41.519885 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:41.520157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] PATH_INFO: `/v3/projects/46dee63e6ef54ae2a037d422a41dacf9` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:41.521500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46dee63e6ef54ae2a037d422a41dacf9)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:41.524707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:41.673335 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46dee63e6ef54ae2a037d422a41dacf9', 'request_id': 'req-89f0df97-f61d-4291-84f4-cd1e8be55f00'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.678984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46dee63e6ef54ae2a037d422a41dacf9', 'request_id': 'req-89f0df97-f61d-4291-84f4-cd1e8be55f00'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.685695 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46dee63e6ef54ae2a037d422a41dacf9', 'request_id': 'req-89f0df97-f61d-4291-84f4-cd1e8be55f00'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.796998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] The token cache is being invalidate because project 46dee63e6ef54ae2a037d422a41dacf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:14:41.797201 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.797807 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-89f0df97-f61d-4291-84f4-cd1e8be55f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:41.799282 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1200/2421] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:14:41 2022] DELETE /identity/v3/projects/46dee63e6ef54ae2a037d422a41dacf9 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:46.594288 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:46.594857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:46.594857 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:46.620815 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] MFA Rules not processed for user `24bee4c00fab42bd8b0cf916e491890d`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:46.622027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:46.688103 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1222/2422] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:14:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:46.998458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afb25-3547-4f09-93bb-fe5f8ba2434c None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:46.998980 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afb25-3547-4f09-93bb-fe5f8ba2434c None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:46.999096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afb25-3547-4f09-93bb-fe5f8ba2434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-1d5afb25-3547-4f09-93bb-fe5f8ba2434c None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:14:47.005760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:14:47.007004 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 11:14:47.007992 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1201/2423] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:14:46 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 9 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:14:48.711461 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:48.717562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:48.760970 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:48.829040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15f78-75d0-4d5b-ab29-9c1c24dae80e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:48.834094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:48.834824 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:48.835102 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:48.835446 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] PATH_INFO: `/v3/users/9a373fc04381459fa1745e46595a0d38` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:48.842584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a373fc04381459fa1745e46595a0d38)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:48.842584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:49.067681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a373fc04381459fa1745e46595a0d38'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:49.068414 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a373fc04381459fa1745e46595a0d38'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:49.199048 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a373fc04381459fa1745e46595a0d38'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:49.200166 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a373fc04381459fa1745e46595a0d38'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:49.214202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a373fc04381459fa1745e46595a0d38'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:49.233558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1223/2424] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:14:48 2022] DELETE /identity/v3/users/9a373fc04381459fa1745e46595a0d38 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:49.238107 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:49.238604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:49.238711 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:49.260116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:49.261405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:49.341003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1202/2425] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:49.349327 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:49.350839 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:49.414246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a58b3-7fb9-4ef9-b20a-ef13351c836b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:49.418082 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6bd36-2456-4438-b66e-09f45948bb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:49.418687 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6bd36-2456-4438-b66e-09f45948bb1d service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:49.418963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6bd36-2456-4438-b66e-09f45948bb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:49.419231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6bd36-2456-4438-b66e-09f45948bb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:49.428266 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6bd36-2456-4438-b66e-09f45948bb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:49.429688 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6bd36-2456-4438-b66e-09f45948bb1d service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:49.447859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1224/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:14:50.064061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:50.065588 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:50.078154 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce45a-4083-4db4-ae18-adbc5f1547e0 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:50.084479 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:50.085122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:50.085403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:50.085707 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] PATH_INFO: `/v3/projects/ccc50abb186448d08d1cc13982bd1a06` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:50.087724 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc50abb186448d08d1cc13982bd1a06)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:50.089913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:50.198102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:50.214063 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc50abb186448d08d1cc13982bd1a06', 'request_id': 'req-bd7ced86-efaa-4922-a88b-c4084681f0f7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:50.220178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc50abb186448d08d1cc13982bd1a06', 'request_id': 'req-bd7ced86-efaa-4922-a88b-c4084681f0f7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:50.228673 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc50abb186448d08d1cc13982bd1a06', 'request_id': 'req-bd7ced86-efaa-4922-a88b-c4084681f0f7'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:50.358694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] The token cache is being invalidate because project ccc50abb186448d08d1cc13982bd1a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:14:50.359427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:50.361323 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-bd7ced86-efaa-4922-a88b-c4084681f0f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:50.367571 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1203/2427] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:14:50 2022] DELETE /identity/v3/projects/ccc50abb186448d08d1cc13982bd1a06 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:50.371282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881704-b882-425c-8ca5-d7fd0e675465 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:50.376745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881704-b882-425c-8ca5-d7fd0e675465 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:50.376745 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e881704-b882-425c-8ca5-d7fd0e675465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:50.401510 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9e881704-b882-425c-8ca5-d7fd0e675465 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:50.403105 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9e881704-b882-425c-8ca5-d7fd0e675465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:50.470348 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:50.492653 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1225/2428] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:50.502897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46778-a128-4de1-bcd8-d729716f20ba None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:50.503510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46778-a128-4de1-bcd8-d729716f20ba None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:50.503982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46778-a128-4de1-bcd8-d729716f20ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:50.534127 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-84d46778-a128-4de1-bcd8-d729716f20ba None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:50.535820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-84d46778-a128-4de1-bcd8-d729716f20ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:50.549318 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1204/2429] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:50.553634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881704-b882-425c-8ca5-d7fd0e675465 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:50.565147 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e881704-b882-425c-8ca5-d7fd0e675465 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:50.570069 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:50.570690 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:50.571163 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:50.571617 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:50.572644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:14:50.573098 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:50.574971 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:50.592742 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d46778-a128-4de1-bcd8-d729716f20ba None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:50.599019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1226/2430] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:14:50.603719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d46778-a128-4de1-bcd8-d729716f20ba None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:50.609271 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:50.610254 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:50.610693 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:50.611114 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:50.611951 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:50.621669 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:50.807732 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1205/2431] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:14:50 2022] POST /identity/v3/projects => generated 383 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:14:50.812050 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:50.826282 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b1b6c-c9e7-4fe2-8f1a-c669a3a3df58 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:50.944142 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:50.944878 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:50.945232 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:50.945575 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:50.946997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:50.949060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:51.053181 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1227/2432] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:14:50 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:14:51.058610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:51.070954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063e8f-311a-4eac-8754-873b1c2403f8 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:51.076754 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:51.077593 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:51.078032 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:51.078306 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:51.078945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:51.080680 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:51.096995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1206/2433] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:14:51.098789 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:51.115366 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41413420-b3f4-4b1b-98d7-dc95fa8dfb2b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:51.125479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11dcf-0232-48a4-a484-be603229b57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:51.125479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11dcf-0232-48a4-a484-be603229b57e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:51.125479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11dcf-0232-48a4-a484-be603229b57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:51.125479 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11dcf-0232-48a4-a484-be603229b57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:51.125890 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a11dcf-0232-48a4-a484-be603229b57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:51.128793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a11dcf-0232-48a4-a484-be603229b57e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:51.143864 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1228/2434] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:14:51.147815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:51.157973 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143b27-36a6-4f9b-84ae-42e344414eef admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:51.162495 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:51.163135 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:51.163411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:51.163681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] PATH_INFO: `/v3/projects/77759e8fc8b14be29e40c404b6b36c5b/users/c690569330344441b079ae22a89c506f/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:51.179084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=77759e8fc8b14be29e40c404b6b36c5b, user_id=c690569330344441b079ae22a89c506f, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:51.183238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:51.251042 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1207/2435] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:14:51 2022] PUT /identity/v3/projects/77759e8fc8b14be29e40c404b6b36c5b/users/c690569330344441b079ae22a89c506f/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:14:51.262412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:51.262785 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:51.263059 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:51.291809 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] MFA Rules not processed for user `c690569330344441b079ae22a89c506f`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:51.293040 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:51.360253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1229/2436] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:14:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:51.371353 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:51.373275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:51.434551 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12611d-7530-4f73-9428-71445eb964fd admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:51.438251 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:51.438856 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:51.439120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:51.439385 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:51.448580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:51.450000 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:51.459642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1208/2437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:14:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:14:56.336256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:56.337834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:56.383936 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:56.403961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e1ae6-1063-443d-9560-a7efedf1f6b8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:56.408695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:56.409543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:56.410169 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:56.410520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] PATH_INFO: `/v3/users/24bee4c00fab42bd8b0cf916e491890d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:56.411698 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24bee4c00fab42bd8b0cf916e491890d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:56.413697 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:56.623619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24bee4c00fab42bd8b0cf916e491890d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:56.624337 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24bee4c00fab42bd8b0cf916e491890d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:56.688215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24bee4c00fab42bd8b0cf916e491890d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:56.688834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24bee4c00fab42bd8b0cf916e491890d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:56.696630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24bee4c00fab42bd8b0cf916e491890d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:56.715619 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1230/2438] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:14:56 2022] DELETE /identity/v3/users/24bee4c00fab42bd8b0cf916e491890d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:56.967307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:56.986203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669d16e-f9a7-48fb-a9a4-c13cbaaa75b6 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:57.035996 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:57.046188 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:57.046822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:57.047102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:57.047375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] PATH_INFO: `/v3/projects/ae8110e8bac044f8a492065e1e06a48d` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:57.048545 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae8110e8bac044f8a492065e1e06a48d)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:57.050448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:57.188609 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae8110e8bac044f8a492065e1e06a48d', 'request_id': 'req-6885a8c3-926a-4983-8669-f21e2a75efe0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.194101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae8110e8bac044f8a492065e1e06a48d', 'request_id': 'req-6885a8c3-926a-4983-8669-f21e2a75efe0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.204087 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae8110e8bac044f8a492065e1e06a48d', 'request_id': 'req-6885a8c3-926a-4983-8669-f21e2a75efe0'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.381026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] The token cache is being invalidate because project ae8110e8bac044f8a492065e1e06a48d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:14:57.381325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.381945 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.383434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1209/2439] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:14:56 2022] DELETE /identity/v3/projects/ae8110e8bac044f8a492065e1e06a48d => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:57.388719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:57.390135 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:57.403598 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c81c9-832b-40a6-81a5-fcaee308d9c7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:57.459057 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:57.459616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:57.459942 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:57.460216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] PATH_INFO: `/v3/users/a0312b8d61eb456c989ba3f57018e38c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:57.461341 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0312b8d61eb456c989ba3f57018e38c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:57.464091 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:57.668950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0312b8d61eb456c989ba3f57018e38c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.669610 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0312b8d61eb456c989ba3f57018e38c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.724240 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0312b8d61eb456c989ba3f57018e38c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.724893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0312b8d61eb456c989ba3f57018e38c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.737254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-732ac57b-1ccb-40f9-9a7f-3630f7acaead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0312b8d61eb456c989ba3f57018e38c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:57.738389 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:57.762728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1231/2440] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:14:57 2022] DELETE /identity/v3/users/a0312b8d61eb456c989ba3f57018e38c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:58.001668 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:58.002906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:58.016242 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885a8c3-926a-4983-8669-f21e2a75efe0 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:58.077473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:58.078102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:58.078384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:58.078655 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] PATH_INFO: `/v3/projects/3782de7bdb2147809bc30f526734ea37` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:58.079766 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3782de7bdb2147809bc30f526734ea37)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:58.081704 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:58.188006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3782de7bdb2147809bc30f526734ea37', 'request_id': 'req-701e1100-39b0-48d9-baf4-673188bd60a1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:58.193744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3782de7bdb2147809bc30f526734ea37', 'request_id': 'req-701e1100-39b0-48d9-baf4-673188bd60a1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:58.200346 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3782de7bdb2147809bc30f526734ea37', 'request_id': 'req-701e1100-39b0-48d9-baf4-673188bd60a1'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:58.293876 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] The token cache is being invalidate because project 3782de7bdb2147809bc30f526734ea37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:14:58.294084 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:58.294706 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:58.296247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1210/2441] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:14:58 2022] DELETE /identity/v3/projects/3782de7bdb2147809bc30f526734ea37 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:03.164651 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e08db-d458-4ccd-a89c-e26d62804a85 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:03.165216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e08db-d458-4ccd-a89c-e26d62804a85 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:03.165216 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e08db-d458-4ccd-a89c-e26d62804a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-5a7e08db-d458-4ccd-a89c-e26d62804a85 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:15:03.172078 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:15:03.173096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Oct 26 11:15:03.174436 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1232/2442] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:15:03 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 10 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:15:03.658384 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:03.659958 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:03.676480 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:03.734698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e1100-39b0-48d9-baf4-673188bd60a1 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:03.739604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:03.740294 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:03.740567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:03.740839 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] PATH_INFO: `/v3/users/3238c1a1ea5149e98c1bc0739a350d0c` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:03.742128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3238c1a1ea5149e98c1bc0739a350d0c)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:03.744106 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:04.029678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3238c1a1ea5149e98c1bc0739a350d0c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:04.060401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3238c1a1ea5149e98c1bc0739a350d0c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:04.062103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3238c1a1ea5149e98c1bc0739a350d0c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:04.069755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3238c1a1ea5149e98c1bc0739a350d0c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:04.070450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3238c1a1ea5149e98c1bc0739a350d0c'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:04.216080 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1211/2443] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:03 2022] DELETE /identity/v3/users/3238c1a1ea5149e98c1bc0739a350d0c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:04.220892 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18680c14-3d41-44df-be66-10886009fe15 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:04.222155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18680c14-3d41-44df-be66-10886009fe15 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:04.222155 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18680c14-3d41-44df-be66-10886009fe15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:04.242427 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-18680c14-3d41-44df-be66-10886009fe15 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:04.243628 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-18680c14-3d41-44df-be66-10886009fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:04.307616 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1233/2444] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:04.318731 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:04.319740 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:04.380392 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad60e2-392b-4b24-962c-c6432693c82f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:04.383968 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:04.384635 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:04.384913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:04.385177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:04.394833 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:04.396381 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:04.406869 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1212/2445] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:15:04.989416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18680c14-3d41-44df-be66-10886009fe15 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:04.991171 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-18680c14-3d41-44df-be66-10886009fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:05.009251 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18680c14-3d41-44df-be66-10886009fe15 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:05.016453 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:05.017275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.017652 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.018024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] PATH_INFO: `/v3/projects/05e0a5b0576842e3993f35dea8c97d67` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.019244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e0a5b0576842e3993f35dea8c97d67)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:05.021311 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:05.101597 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:05.103233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:05.116198 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fc362-8435-4f5b-a2bb-65b12dd5c086 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:05.121104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:05.121739 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.122018 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.122291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] PATH_INFO: `/v3/users/c690569330344441b079ae22a89c506f` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.124913 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c690569330344441b079ae22a89c506f)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:05.127157 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:05.216758 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e0a5b0576842e3993f35dea8c97d67', 'request_id': 'req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.223696 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e0a5b0576842e3993f35dea8c97d67', 'request_id': 'req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.230834 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e0a5b0576842e3993f35dea8c97d67', 'request_id': 'req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.370711 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] The token cache is being invalidate because project 05e0a5b0576842e3993f35dea8c97d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:05.371252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.372294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a26845f4-0b0d-4e91-842e-b3d84a32f6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.376751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1234/2446] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:04 2022] DELETE /identity/v3/projects/05e0a5b0576842e3993f35dea8c97d67 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:05.383123 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee8a1a-4a84-4c74-9c44-696b488648ec None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.383618 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee8a1a-4a84-4c74-9c44-696b488648ec None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.384145 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee8a1a-4a84-4c74-9c44-696b488648ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.405395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:05.418630 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2dee8a1a-4a84-4c74-9c44-696b488648ec None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:05.419964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2dee8a1a-4a84-4c74-9c44-696b488648ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:05.434405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c690569330344441b079ae22a89c506f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.457387 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c690569330344441b079ae22a89c506f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.458403 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c690569330344441b079ae22a89c506f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.469762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c690569330344441b079ae22a89c506f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.470768 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-1f9de299-23ca-4bad-a974-fdc131329c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c690569330344441b079ae22a89c506f'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:05.521138 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1235/2447] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:05.529448 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a4e4-7456-4fc2-8b5d-26cd38e3fd1e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.530025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a4e4-7456-4fc2-8b5d-26cd38e3fd1e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.530025 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a4e4-7456-4fc2-8b5d-26cd38e3fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.542107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:05.554499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c788a4e4-7456-4fc2-8b5d-26cd38e3fd1e None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:05.556452 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c788a4e4-7456-4fc2-8b5d-26cd38e3fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:05.565873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1236/2448] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:15:05.571158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788a4e4-7456-4fc2-8b5d-26cd38e3fd1e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:05.582116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1213/2449] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:05 2022] DELETE /identity/v3/users/c690569330344441b079ae22a89c506f => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:15:05.584006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.584345 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.584824 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.586390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788a4e4-7456-4fc2-8b5d-26cd38e3fd1e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:05.610923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:05.612298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:05.684874 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:05.685563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.685863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.686725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.690950 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:15:05.691417 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:05.697051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:05.715677 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1214/2450] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:15:05.728832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1237/2451] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:15:05.729732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:05.730177 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:05.734995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:05.748483 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c073466-f8ad-41c3-bc9c-e2934efed081 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:05.754828 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:05.755451 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.755735 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.763899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.763899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:05.763899 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:05.836577 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4386a15-9d15-4611-88b1-7d65a682443a None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:05.840337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:05.841259 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:05.841755 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:05.842901 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:05.855470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:05.858774 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:05.879233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1215/2452] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:05.911961 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1238/2453] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:15:05 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:05.916235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:05.931747 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26fe4-e3b6-4e43-9c90-0dd201c42de0 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:06.048180 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:06.048966 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.049292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.049568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.050233 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:06.052238 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:06.233312 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1216/2454] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:15:05 2022] POST /identity/v3/users => generated 293 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:06.237495 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:06.248520 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b102e-01e2-44d5-9de8-09af75e49a0e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:06.257801 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:06.258454 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.258736 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.259005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.260392 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:06.261826 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:06.277919 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1239/2455] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:15:06.278978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:06.290085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d852962-f680-4f93-9d2d-9b46ba63e85e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:06.295422 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:06.296373 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.296702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.296944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.297583 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:06.299183 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:06.309942 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1217/2456] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:15:06.313966 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:06.329153 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca00362-dbb9-42a5-96e3-eba3ae68f12a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:06.334107 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6a97-33cf-4151-88d6-f105a3efaef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:06.334817 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6a97-33cf-4151-88d6-f105a3efaef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.335092 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6a97-33cf-4151-88d6-f105a3efaef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.335402 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6a97-33cf-4151-88d6-f105a3efaef0 admin admin] PATH_INFO: `/v3/projects/564b15ef544147a383f39bbfda4a1ecd/users/a85ddee9238c43769a53438d29d981dd/roles/b109a364aa914a98aee2e78986065def` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.351960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6a97-33cf-4151-88d6-f105a3efaef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564b15ef544147a383f39bbfda4a1ecd, user_id=a85ddee9238c43769a53438d29d981dd, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:06.354709 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6a97-33cf-4151-88d6-f105a3efaef0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:06.435904 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:06.438305 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:06.454230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:06.459684 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1240/2457] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:15:06 2022] PUT /identity/v3/projects/564b15ef544147a383f39bbfda4a1ecd/users/a85ddee9238c43769a53438d29d981dd/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:06.464978 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.465320 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.465512 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.469162 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c35d81-e50e-4904-9311-d56e38d0e066 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:06.496507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:06.497947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:06.542802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:06.543443 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.543727 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.544036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] PATH_INFO: `/v3/projects/77759e8fc8b14be29e40c404b6b36c5b` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.545217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77759e8fc8b14be29e40c404b6b36c5b)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:06.547279 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:06.559503 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1241/2458] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:15:06.570005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:06.571695 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:06.643028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f4bff-c256-4a12-a0b7-d05629f2a6b6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:06.646741 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b4d3a-61e4-4efc-b739-001502b6e0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:06.647412 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b4d3a-61e4-4efc-b739-001502b6e0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.647644 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b4d3a-61e4-4efc-b739-001502b6e0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.647947 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b4d3a-61e4-4efc-b739-001502b6e0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.657585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b4d3a-61e4-4efc-b739-001502b6e0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:06.659209 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b4d3a-61e4-4efc-b739-001502b6e0a3 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:06.669740 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1242/2459] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:15:06.716349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77759e8fc8b14be29e40c404b6b36c5b', 'request_id': 'req-923ad047-22d1-4225-a297-3d1bfa93ac75'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:06.723232 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77759e8fc8b14be29e40c404b6b36c5b', 'request_id': 'req-923ad047-22d1-4225-a297-3d1bfa93ac75'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:06.742130 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77759e8fc8b14be29e40c404b6b36c5b', 'request_id': 'req-923ad047-22d1-4225-a297-3d1bfa93ac75'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:06.901463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] The token cache is being invalidate because project 77759e8fc8b14be29e40c404b6b36c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:06.901757 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:06.902416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-923ad047-22d1-4225-a297-3d1bfa93ac75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:06.904104 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1218/2460] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:06 2022] DELETE /identity/v3/projects/77759e8fc8b14be29e40c404b6b36c5b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:06.910607 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7a428-8409-4fcd-baf5-ef217286b9ce None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:06.910896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7a428-8409-4fcd-baf5-ef217286b9ce None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:06.911219 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7a428-8409-4fcd-baf5-ef217286b9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:06.930814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bfd7a428-8409-4fcd-baf5-ef217286b9ce None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:06.932051 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bfd7a428-8409-4fcd-baf5-ef217286b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:06.997314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1243/2461] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:07.005448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bda740-2e5e-4272-a735-4721e466c4d9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.005735 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bda740-2e5e-4272-a735-4721e466c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.006036 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bda740-2e5e-4272-a735-4721e466c4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.026887 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-90bda740-2e5e-4272-a735-4721e466c4d9 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:07.028125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-90bda740-2e5e-4272-a735-4721e466c4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:07.048341 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1219/2462] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:07.053492 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7a428-8409-4fcd-baf5-ef217286b9ce None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.065631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7a428-8409-4fcd-baf5-ef217286b9ce None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.071728 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.072429 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.072714 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.072991 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.073679 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:15:07.073985 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:07.075596 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:07.102101 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1244/2463] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:15:07.106762 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bda740-2e5e-4272-a735-4721e466c4d9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.113587 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:07.122941 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bda740-2e5e-4272-a735-4721e466c4d9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.147954 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.148861 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.150349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.150938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.152299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:07.156619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:07.253581 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.256027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:07.271128 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db72401-2b65-4950-9781-a73220aed533 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.276603 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.277793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.277917 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.278100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] PATH_INFO: `/v3/users/35beeaac0d01404597acefb86654bea2` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.279486 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35beeaac0d01404597acefb86654bea2)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:07.281693 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:07.364771 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1220/2464] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:15:07 2022] POST /identity/v3/projects => generated 375 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:15:07.367330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.440420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bc3d1-5921-4e76-860e-29592f9702fb admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.445033 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.445638 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.445916 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.446186 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.446806 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:07.448843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:07.574917 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1221/2465] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:15:07 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:07.579553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.589534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b8929-2fc2-4dd2-acdc-c11ee041d36d admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.594522 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.595089 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.595365 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.595633 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.596308 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:07.597897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:07.610580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1222/2466] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:15:07.615176 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.625943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35beeaac0d01404597acefb86654bea2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:07.626873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35beeaac0d01404597acefb86654bea2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:07.630447 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aa0cc-5c85-4e5d-b533-c5304af44b1b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.636628 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.636950 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.637116 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.637416 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.638453 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:07.640786 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:07.659640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1223/2467] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:15:07.663462 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.674974 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7514601-9d68-4fa7-b640-9f28191aea75 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.681360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.682035 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.682344 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.682624 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] PATH_INFO: `/v3/projects/dde3823afe64468db10f0141d8735a7e/users/72b52dd0fa2848e6ab2b64151f4d54fd/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.700848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde3823afe64468db10f0141d8735a7e, user_id=72b52dd0fa2848e6ab2b64151f4d54fd, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:07.704171 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:07.719058 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35beeaac0d01404597acefb86654bea2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:07.719701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35beeaac0d01404597acefb86654bea2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:07.728246 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bdfb81d1-e4ec-4555-846e-4c9424264f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35beeaac0d01404597acefb86654bea2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:07.751287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1245/2468] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:07 2022] DELETE /identity/v3/users/35beeaac0d01404597acefb86654bea2 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:15:07.756701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab01b5-fea4-4bcb-8a19-245d9ed32af4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.757019 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab01b5-fea4-4bcb-8a19-245d9ed32af4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.761087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab01b5-fea4-4bcb-8a19-245d9ed32af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.779242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2bab01b5-fea4-4bcb-8a19-245d9ed32af4 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:07.781046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2bab01b5-fea4-4bcb-8a19-245d9ed32af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:07.828760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1224/2469] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:15:07 2022] PUT /identity/v3/projects/dde3823afe64468db10f0141d8735a7e/users/72b52dd0fa2848e6ab2b64151f4d54fd/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:15:07.845853 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1246/2470] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:07.860555 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:07.879349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:07.910953 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.911303 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.911519 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:07.957681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cec281-d85f-4991-8c52-23f76d047773 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:07.960925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] MFA Rules not processed for user `72b52dd0fa2848e6ab2b64151f4d54fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:07.961778 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:07.962294 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:07.962475 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:07.962984 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:07.963108 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:08.038424 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:08.039607 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:08.055186 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1247/2471] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:08.061837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1225/2472] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:08.064260 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:08.065902 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:08.136100 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f0713-e261-4185-819c-5b8629183426 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:08.140743 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4091b8c-8e84-4397-9e8e-1784f66c3e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:08.141350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4091b8c-8e84-4397-9e8e-1784f66c3e1a service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:08.141625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4091b8c-8e84-4397-9e8e-1784f66c3e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:08.141893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4091b8c-8e84-4397-9e8e-1784f66c3e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:08.149444 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:08.152060 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4091b8c-8e84-4397-9e8e-1784f66c3e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:08.153778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4091b8c-8e84-4397-9e8e-1784f66c3e1a service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:08.163276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1248/2473] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:08.277623 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:09.063006 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:09.065490 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:09.085389 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2a1ba-9bcd-4cc3-9299-e0b24a2f1a26 service neutron] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:09.091330 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:09.092280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:09.092926 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:09.093420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] PATH_INFO: `/v3/projects/3cea63489e004c349fd955adc32c4400` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:09.095247 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cea63489e004c349fd955adc32c4400)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:09.097563 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:09.244334 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cea63489e004c349fd955adc32c4400', 'request_id': 'req-531a9564-6cdf-41f9-a74d-5e4c7187386b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:09.252562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cea63489e004c349fd955adc32c4400', 'request_id': 'req-531a9564-6cdf-41f9-a74d-5e4c7187386b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:09.262488 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cea63489e004c349fd955adc32c4400', 'request_id': 'req-531a9564-6cdf-41f9-a74d-5e4c7187386b'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:09.397871 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] The token cache is being invalidate because project 3cea63489e004c349fd955adc32c4400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:09.398367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:09.400244 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-531a9564-6cdf-41f9-a74d-5e4c7187386b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:09.416507 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1226/2474] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:09 2022] DELETE /identity/v3/projects/3cea63489e004c349fd955adc32c4400 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:15:09.418578 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20b079-7a8c-46c3-85f3-93f0b842ff87 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:09.419136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20b079-7a8c-46c3-85f3-93f0b842ff87 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:09.419136 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20b079-7a8c-46c3-85f3-93f0b842ff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:09.447943 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5c20b079-7a8c-46c3-85f3-93f0b842ff87 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:09.450139 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5c20b079-7a8c-46c3-85f3-93f0b842ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:09.550893 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1249/2475] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:09.552349 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74d54-1cf9-437c-b5c6-3abc97737ca1 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:09.552648 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74d54-1cf9-437c-b5c6-3abc97737ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:09.553026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e74d54-1cf9-437c-b5c6-3abc97737ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:09.578401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-55e74d54-1cf9-437c-b5c6-3abc97737ca1 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:09.579752 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-55e74d54-1cf9-437c-b5c6-3abc97737ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:09.596160 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20b079-7a8c-46c3-85f3-93f0b842ff87 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:09.602474 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1227/2476] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:15:09.611594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20b079-7a8c-46c3-85f3-93f0b842ff87 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:09.617458 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:09.618526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:09.619253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:09.619253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:09.619588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:15:09.620175 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:09.621928 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:09.651291 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74d54-1cf9-437c-b5c6-3abc97737ca1 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:09.651816 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1250/2477] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:15:09.663052 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e74d54-1cf9-437c-b5c6-3abc97737ca1 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:09.668325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d8d51-9973-4092-8f07-8172a3cf41dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:09.668952 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d8d51-9973-4092-8f07-8172a3cf41dd admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:09.669235 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d8d51-9973-4092-8f07-8172a3cf41dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:09.669579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d8d51-9973-4092-8f07-8172a3cf41dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:09.670218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d8d51-9973-4092-8f07-8172a3cf41dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:09.672500 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d8d51-9973-4092-8f07-8172a3cf41dd admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:09.803094 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:09.804505 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1228/2478] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:15:09 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:09.839550 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd344e0-80cb-4646-bb54-a5b87a0fb719 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:09.920200 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:09.921848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:09.921848 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:09.922206 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:09.922855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:09.925028 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:09.958562 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f4a0-4f25-432a-8108-4558c7628ff9 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:09.958881 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f4a0-4f25-432a-8108-4558c7628ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:09.959441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1f4a0-4f25-432a-8108-4558c7628ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:09.985452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-14d1f4a0-4f25-432a-8108-4558c7628ff9 None None] MFA Rules not processed for user `72b52dd0fa2848e6ab2b64151f4d54fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:09.985452 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-14d1f4a0-4f25-432a-8108-4558c7628ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:10.013930 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1251/2479] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:15:09 2022] POST /identity/v3/users => generated 282 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:10.015600 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:10.030526 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b54272-8119-4983-8838-8365542f0fbb admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:10.043044 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:10.043669 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:10.043976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:10.044255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:10.045144 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:10.051197 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:10.078678 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1252/2480] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:15:10.082467 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:10.094336 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1229/2481] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:15:10.100449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e505d88-9d13-4474-9045-ac86ee298522 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:10.109401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee8543-132a-4002-9634-53b15551c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:10.111211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee8543-132a-4002-9634-53b15551c834 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:10.111987 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee8543-132a-4002-9634-53b15551c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:10.112273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee8543-132a-4002-9634-53b15551c834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:10.112931 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee8543-132a-4002-9634-53b15551c834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:10.115310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee8543-132a-4002-9634-53b15551c834 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:10.133178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1f4a0-4f25-432a-8108-4558c7628ff9 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:10.144802 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1253/2482] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:15:10.149401 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1f4a0-4f25-432a-8108-4558c7628ff9 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:10.156644 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:10.157476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:10.157760 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:10.158037 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] PATH_INFO: `/v3/projects/0a4baf33753b4691b5463392f5b2ab59/users/6ebf625256b34f27b1b2a29e2ba0cc4d/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:10.184440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4baf33753b4691b5463392f5b2ab59, user_id=6ebf625256b34f27b1b2a29e2ba0cc4d, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:10.188218 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:10.307934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1230/2483] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:15:10 2022] PUT /identity/v3/projects/0a4baf33753b4691b5463392f5b2ab59/users/6ebf625256b34f27b1b2a29e2ba0cc4d/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:10.318821 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:10.319507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:10.320022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:10.353792 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] MFA Rules not processed for user `6ebf625256b34f27b1b2a29e2ba0cc4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:10.356151 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:10.468560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:10.476084 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1254/2484] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:10.492450 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:10.494473 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:10.588820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:10.620382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bff452-97dd-441d-91e0-8e7aaacdbe0a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:10.625125 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:10.625812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:10.626096 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:10.626561 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:10.640661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:10.641432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:10.653206 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1231/2485] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:11.196545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:11.198352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:11.273965 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:11.326838 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d16e63-e154-43ed-9bbd-3a1620af828a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:11.353859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be29425-b217-4811-b1c8-a43cd8e716e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c154983aff14d9895fb4b542db61e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:11.353859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be29425-b217-4811-b1c8-a43cd8e716e5 service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:11.353859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be29425-b217-4811-b1c8-a43cd8e716e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:11.353859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be29425-b217-4811-b1c8-a43cd8e716e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:11.354609 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3be29425-b217-4811-b1c8-a43cd8e716e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:11.370750 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be29425-b217-4811-b1c8-a43cd8e716e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:11.374004 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be29425-b217-4811-b1c8-a43cd8e716e5 service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:11.392179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1255/2486] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:13.447744 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:13.450024 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:13.464005 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:13.580463 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7af4e-2454-476e-a6e8-89417cbc4853 service cinder] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:13.590745 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:13.591667 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:13.592566 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:13.593159 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] PATH_INFO: `/v3/users/14327353a11e4027bde6c57b923eb185` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:13.597567 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14327353a11e4027bde6c57b923eb185)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:13.607369 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:13.883717 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14327353a11e4027bde6c57b923eb185'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:13.903738 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14327353a11e4027bde6c57b923eb185'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:13.904575 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14327353a11e4027bde6c57b923eb185'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:13.917873 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14327353a11e4027bde6c57b923eb185'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:13.918925 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14327353a11e4027bde6c57b923eb185'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:14.006086 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1232/2487] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:13 2022] DELETE /identity/v3/users/14327353a11e4027bde6c57b923eb185 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:14.042583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:14.046088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:14.046088 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:14.084433 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] MFA Rules not processed for user `a85ddee9238c43769a53438d29d981dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:14.085814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:14.239252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1256/2488] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:14.242102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:14.242102 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:14.390005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:14.391700 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:14.396295 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443211d1-7862-494b-adb2-f73eeafac112 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:14.401470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:14.401944 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:14.402439 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:14.402582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:14.403354 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:14.406286 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:14.418192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b82af-5499-40fe-accb-fd10c60c2c59 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:14.428192 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:14.429496 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:14.429778 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:14.430405 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] PATH_INFO: `/v3/projects/4a9ff4fa4ae34163b8e9ea42fae1d723` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:14.431586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9ff4fa4ae34163b8e9ea42fae1d723)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:14.433960 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:14.543874 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1233/2489] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:15:14 2022] POST /identity/v3/projects => generated 383 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:14.548547 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:14.585404 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9ff4fa4ae34163b8e9ea42fae1d723', 'request_id': 'req-16f17527-4fe2-4b48-b538-349a9d77d3b5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:14.593800 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9ff4fa4ae34163b8e9ea42fae1d723', 'request_id': 'req-16f17527-4fe2-4b48-b538-349a9d77d3b5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:14.605545 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9ff4fa4ae34163b8e9ea42fae1d723', 'request_id': 'req-16f17527-4fe2-4b48-b538-349a9d77d3b5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:14.629057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb1c1b-23a6-47ba-b653-e2a923f25e5e admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:14.696595 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] The token cache is being invalidate because project 4a9ff4fa4ae34163b8e9ea42fae1d723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:14.697314 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:14.697650 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:14.702582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1257/2490] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:14 2022] DELETE /identity/v3/projects/4a9ff4fa4ae34163b8e9ea42fae1d723 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:14.704224 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:14.705729 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:14.710864 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:14.711460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:14.711934 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:14.712214 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] PATH_INFO: `/v3/users` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:14.712854 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:14.714848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:14.724174 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f17527-4fe2-4b48-b538-349a9d77d3b5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:14.730134 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:14.731047 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:14.731406 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:14.731751 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] PATH_INFO: `/v3/users/39493a0141f54214801f3291b20f8dc3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:14.733811 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] RBAC: Authorizing `identity:delete_user(user_id=39493a0141f54214801f3291b20f8dc3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:14.736911 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:14.866217 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1234/2491] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:15:14 2022] POST /identity/v3/users => generated 298 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:15:14.869888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:14.872128 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:14.891360 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d30ad-bc18-46d3-bc13-453597ef4040 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:14.899103 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:14.899802 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:14.900120 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:14.900397 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:14.901163 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:14.902820 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:14.914703 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1235/2492] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:15:14.918848 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:14.930897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e562-86e5-4a02-bc8b-d118909c23be admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:14.936702 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:14.938767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:14.938767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:14.938767 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] PATH_INFO: `/v3/projects/14332372c0304c00b26b8b97ff95f816/users/432c42f83107420db971b035047b52f2/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:14.991940 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14332372c0304c00b26b8b97ff95f816, user_id=432c42f83107420db971b035047b52f2, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:14.995082 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:15.009275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39493a0141f54214801f3291b20f8dc3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.010255 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39493a0141f54214801f3291b20f8dc3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.015214 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:15.121436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1236/2493] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:15:14 2022] PUT /identity/v3/projects/14332372c0304c00b26b8b97ff95f816/users/432c42f83107420db971b035047b52f2/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:15:15.121436 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:15.146896 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39493a0141f54214801f3291b20f8dc3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.148229 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39493a0141f54214801f3291b20f8dc3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.157525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39493a0141f54214801f3291b20f8dc3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.179832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1258/2494] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:14 2022] DELETE /identity/v3/users/39493a0141f54214801f3291b20f8dc3 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:15:15.193396 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7e9ca-4156-4dc5-8f27-d1092224943b admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:15.283181 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:15.283818 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:15.284122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:15.284391 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:15.285027 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:15.286626 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:15.304842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1237/2495] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:15:15.306218 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:15.307704 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:15.333626 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ae8d-6823-4ea4-92f1-946f7a56d81a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:15.346770 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db543e7-5282-49ac-a865-a37a3a30e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:15.347661 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db543e7-5282-49ac-a865-a37a3a30e0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:15.351095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db543e7-5282-49ac-a865-a37a3a30e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:15.351095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db543e7-5282-49ac-a865-a37a3a30e0e9 admin admin] PATH_INFO: `/v3/projects/14332372c0304c00b26b8b97ff95f816/users/432c42f83107420db971b035047b52f2/roles/25ab652070944ce094aac8e7e6ae0d77` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:15.351095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db543e7-5282-49ac-a865-a37a3a30e0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14332372c0304c00b26b8b97ff95f816, user_id=432c42f83107420db971b035047b52f2, role_id=25ab652070944ce094aac8e7e6ae0d77)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:15.354469 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db543e7-5282-49ac-a865-a37a3a30e0e9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:15.368708 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1259/2496] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:15:15 2022] PUT /identity/v3/projects/14332372c0304c00b26b8b97ff95f816/users/432c42f83107420db971b035047b52f2/roles/25ab652070944ce094aac8e7e6ae0d77 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:15.555647 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:15.567694 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:15.590579 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07939508-f008-4cd1-8166-5c98a61e5709 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:15.698604 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:15.699258 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:15.699621 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:15.699929 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] PATH_INFO: `/v3/projects/23ac2b9dc837463fbb94cac1140ad915` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:15.708484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ac2b9dc837463fbb94cac1140ad915)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:15.708484 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:15.883531 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ac2b9dc837463fbb94cac1140ad915', 'request_id': 'req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.906595 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ac2b9dc837463fbb94cac1140ad915', 'request_id': 'req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.915212 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ac2b9dc837463fbb94cac1140ad915', 'request_id': 'req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.933411 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:15.994729 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] The token cache is being invalidate because project 23ac2b9dc837463fbb94cac1140ad915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:15.997002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.997002 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:15.997493 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1238/2497] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:15 2022] DELETE /identity/v3/projects/23ac2b9dc837463fbb94cac1140ad915 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:18.393553 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:20.800379 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c039-996a-412c-87c1-c24523118970 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:20.800891 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c039-996a-412c-87c1-c24523118970 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:20.800976 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c039-996a-412c-87c1-c24523118970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:20.844222 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6e17c039-996a-412c-87c1-c24523118970 None None] MFA Rules not processed for user `ce0cb1d976784ea6af783fd193ddac20`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:20.845449 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6e17c039-996a-412c-87c1-c24523118970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:20.909286 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1260/2498] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Oct 26 11:15:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:20.922664 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:20.925095 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:20.965085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892ed04-5d5f-46f2-9e9e-916f0edb01c9 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:20.969927 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458b491-e38f-4bca-93d3-cf4dfe3608f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:20.970536 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458b491-e38f-4bca-93d3-cf4dfe3608f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:20.970986 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458b491-e38f-4bca-93d3-cf4dfe3608f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:20.971429 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458b491-e38f-4bca-93d3-cf4dfe3608f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:20.979893 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458b491-e38f-4bca-93d3-cf4dfe3608f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:20.981568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458b491-e38f-4bca-93d3-cf4dfe3608f7 service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:20.993642 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1239/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:22.082901 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d3e3d-5461-4648-9c1a-99258a13549c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:22.083371 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d3e3d-5461-4648-9c1a-99258a13549c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:22.083491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d3e3d-5461-4648-9c1a-99258a13549c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:22.085441 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1261/2500] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 26 11:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:15:22.096843 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299cf16-ce57-48f3-942d-b468d9b1a226 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:22.097444 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299cf16-ce57-48f3-942d-b468d9b1a226 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:22.097708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299cf16-ce57-48f3-942d-b468d9b1a226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:22.131099 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-f299cf16-ce57-48f3-942d-b468d9b1a226 None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:22.132923 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-f299cf16-ce57-48f3-942d-b468d9b1a226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:22.176064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:22.232268 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1240/2501] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 26 11:15:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:22.244995 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d3e3d-5461-4648-9c1a-99258a13549c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:22.247332 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d26d3e3d-5461-4648-9c1a-99258a13549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:22.270254 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d3e3d-5461-4648-9c1a-99258a13549c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:22.277176 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:22.277813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:22.278108 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:22.278374 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:22.289562 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:22.290117 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:22.302354 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1262/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:22.359580 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab7fb1-4bfa-4a01-9908-e54c0a1d7097 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:22.361062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab7fb1-4bfa-4a01-9908-e54c0a1d7097 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:22.361468 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab7fb1-4bfa-4a01-9908-e54c0a1d7097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:22.399433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-a5ab7fb1-4bfa-4a01-9908-e54c0a1d7097 None None] MFA Rules not processed for user `432c42f83107420db971b035047b52f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:22.400713 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-a5ab7fb1-4bfa-4a01-9908-e54c0a1d7097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:22.495880 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1241/2503] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:22.505346 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:22.506759 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:22.569202 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cee54-c81d-4066-8634-207ea60c0984 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:22.576383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9627e6a9e86e43b694f723bd4109460e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:22.576383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:22.576383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:22.576383 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:22.577680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:22.638027 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:22.642793 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:22.644287 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:22.654583 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1263/2504] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:22.992827 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c297796-c779-455b-b55c-d139f002235f None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:22.993122 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c297796-c779-455b-b55c-d139f002235f None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:22.993398 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c297796-c779-455b-b55c-d139f002235f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:23.020178 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-5c297796-c779-455b-b55c-d139f002235f None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:23.021375 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5c297796-c779-455b-b55c-d139f002235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:23.082009 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1242/2505] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:15:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:23.339350 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:23.340798 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:23.416264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a5b8-d748-4063-ada0-7a04fa6d3af3 service glance] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:23.422179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:23.422922 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:23.423201 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:23.423470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:23.432018 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:23.433586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:23.441863 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1264/2506] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:15:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:15:24.092003 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c297796-c779-455b-b55c-d139f002235f None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:24.103697 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c297796-c779-455b-b55c-d139f002235f None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:24.107994 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a786462-b8a7-4672-8131-7c35e53f6bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:24.109678 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a786462-b8a7-4672-8131-7c35e53f6bfb service neutron] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:24.116280 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a786462-b8a7-4672-8131-7c35e53f6bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:24.117685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a786462-b8a7-4672-8131-7c35e53f6bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:24.119292 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-5a786462-b8a7-4672-8131-7c35e53f6bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:24.138822 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a786462-b8a7-4672-8131-7c35e53f6bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:24.142091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a786462-b8a7-4672-8131-7c35e53f6bfb service neutron] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:24.169058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1243/2507] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:25.613005 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:25.807659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b253b0d-3e8d-448e-9099-6b41494e2e1d service swift] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:25.807659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a4ce2-217f-4cc4-8e41-0ed17030a1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:25.807659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a4ce2-217f-4cc4-8e41-0ed17030a1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:25.807659 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a4ce2-217f-4cc4-8e41-0ed17030a1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:25.815625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a4ce2-217f-4cc4-8e41-0ed17030a1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:25.815625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-856a4ce2-217f-4cc4-8e41-0ed17030a1b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:25.815625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a4ce2-217f-4cc4-8e41-0ed17030a1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:25.815625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a4ce2-217f-4cc4-8e41-0ed17030a1b4 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:25.815625 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1265/2508] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:32.493721 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f3d80-af1c-4d9f-8053-531dc64d64d7 None None] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:32.494366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f3d80-af1c-4d9f-8053-531dc64d64d7 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:32.494366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f3d80-af1c-4d9f-8053-531dc64d64d7 None None] PATH_INFO: `/` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:32.495922 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1244/2509] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 26 11:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:15:32.512794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7386a23-036f-475d-9f78-3d289066e9a7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:32.513353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7386a23-036f-475d-9f78-3d289066e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:32.513353 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7386a23-036f-475d-9f78-3d289066e9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:32.538244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a7386a23-036f-475d-9f78-3d289066e9a7 None None] MFA Rules not processed for user `c3d05bdc4c9b4f24b1d58ef157b32ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:32.543832 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a7386a23-036f-475d-9f78-3d289066e9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:32.555507 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1266/2510] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 26 11:15:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:32.568812 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f3d80-af1c-4d9f-8053-531dc64d64d7 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:32.571079 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-528f3d80-af1c-4d9f-8053-531dc64d64d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:32.616076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f3d80-af1c-4d9f-8053-531dc64d64d7 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:32.625230 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f249c1-8c1d-4d4f-aff6-eb522e58645f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3d05bdc4c9b4f24b1d58ef157b32ae0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:32.626564 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f249c1-8c1d-4d4f-aff6-eb522e58645f service nova] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:32.626851 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f249c1-8c1d-4d4f-aff6-eb522e58645f service nova] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:32.627129 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f249c1-8c1d-4d4f-aff6-eb522e58645f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:32.641423 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f249c1-8c1d-4d4f-aff6-eb522e58645f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:32.643508 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f249c1-8c1d-4d4f-aff6-eb522e58645f service nova] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:32.656813 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1245/2511] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:35.450020 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71d05f-66b8-4913-8666-e2f101dbd57c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:35.450646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71d05f-66b8-4913-8666-e2f101dbd57c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:35.450646 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71d05f-66b8-4913-8666-e2f101dbd57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:35.470712 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-de71d05f-66b8-4913-8666-e2f101dbd57c None None] MFA Rules not processed for user `a85ddee9238c43769a53438d29d981dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:35.471963 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-de71d05f-66b8-4913-8666-e2f101dbd57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:35.478794 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1267/2512] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:35.793460 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed61f-ad6c-4ee6-b3bd-3d65eb6e6cbf None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:35.794045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed61f-ad6c-4ee6-b3bd-3d65eb6e6cbf None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:35.794045 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed61f-ad6c-4ee6-b3bd-3d65eb6e6cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application [None req-afeed61f-ad6c-4ee6-b3bd-3d65eb6e6cbf None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:15:35.801327 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application return f(*args, **kwargs) Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Oct 26 11:15:35.802438 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: ERROR keystone.server.flask.application Oct 26 11:15:35.803622 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1246/2513] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:15:35 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 10 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Oct 26 11:15:35.931686 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7469b6-cd93-42f7-9d13-da71410f3277 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:35.932030 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7469b6-cd93-42f7-9d13-da71410f3277 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:35.932310 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7469b6-cd93-42f7-9d13-da71410f3277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:35.956359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4a7469b6-cd93-42f7-9d13-da71410f3277 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:35.956359 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4a7469b6-cd93-42f7-9d13-da71410f3277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:35.960295 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1268/2514] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 26 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:35.974467 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeed61f-ad6c-4ee6-b3bd-3d65eb6e6cbf None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:35.989709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeed61f-ad6c-4ee6-b3bd-3d65eb6e6cbf None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:35.994619 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:35.995211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:35.995491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:35.995765 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:36.004393 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:36.006194 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:36.015275 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1247/2515] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:15:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:15:41.650731 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdfc92-5983-4ffa-bac5-eb725792e063 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:41.651253 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdfc92-5983-4ffa-bac5-eb725792e063 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:41.651370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdfc92-5983-4ffa-bac5-eb725792e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:41.672873 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-14cdfc92-5983-4ffa-bac5-eb725792e063 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:41.674137 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-14cdfc92-5983-4ffa-bac5-eb725792e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:41.682431 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1269/2516] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:15:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:41.695143 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:41.705892 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb0764-a958-4dc1-8b2c-ff2628c75d4f service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:41.711521 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:41.712182 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:41.712448 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:41.712987 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:41.722337 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:41.724064 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:41.729978 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=100 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:41.734067 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1248/2517] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:15:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:15:42.972062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03021b7-06f6-4321-bf2e-82cab543de5e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:42.972634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03021b7-06f6-4321-bf2e-82cab543de5e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:42.972634 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03021b7-06f6-4321-bf2e-82cab543de5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:43.000599 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c03021b7-06f6-4321-bf2e-82cab543de5e None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:43.002120 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c03021b7-06f6-4321-bf2e-82cab543de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:43.010718 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1270/2518] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 26 11:15:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:43.025501 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:43.046756 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a9931-c52a-46b4-9f67-e6cfb39b6927 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:43.052924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:43.053554 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:43.054026 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:43.054307 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:43.065656 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:43.067471 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:43.078069 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1249/2519] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:15:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:15:44.009530 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48211-0346-4c65-9872-7f845df10115 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:44.009998 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48211-0346-4c65-9872-7f845df10115 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:44.010256 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48211-0346-4c65-9872-7f845df10115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:44.031772 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d3c48211-0346-4c65-9872-7f845df10115 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:44.033170 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d3c48211-0346-4c65-9872-7f845df10115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:44.045813 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1271/2520] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 26 11:15:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:15:44.058720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:44.070142 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b4f0-3559-40cf-86f9-8172c6894406 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:44.075486 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:44.076165 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:44.076440 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:44.076708 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:44.085877 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:44.087441 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:44.095582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1250/2521] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:15:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:15:44.871672 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:44.872215 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:44.872701 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:44.900797 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] MFA Rules not processed for user `d64e1522d1d4425bbb88d78e4f8863d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:44.902011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:44.909594 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1272/2522] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 26 11:15:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:44.922775 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:44.933763 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5f2bf-95c0-4106-ad88-b6d0aff42008 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:44.939510 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb4908b26e64c538c2b573f3cf26bf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:44.940189 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:44.940469 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:44.940736 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:44.949414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:44.953998 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:44.960057 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1251/2523] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 26 11:15:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:15:45.189760 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:45.191979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:45.259211 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda1018-3576-44b4-bf5d-484375060ed8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:45.263905 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:45.265352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:45.265352 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:45.266001 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] PATH_INFO: `/v3/users/a85ddee9238c43769a53438d29d981dd` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:45.267180 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a85ddee9238c43769a53438d29d981dd)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:45.269131 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:45.382732 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:45.493491 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a85ddee9238c43769a53438d29d981dd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:45.494183 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a85ddee9238c43769a53438d29d981dd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:45.570264 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a85ddee9238c43769a53438d29d981dd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:45.570888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a85ddee9238c43769a53438d29d981dd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:45.578753 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a85ddee9238c43769a53438d29d981dd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:45.601812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1273/2524] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:45 2022] DELETE /identity/v3/users/a85ddee9238c43769a53438d29d981dd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:45.846169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:45.860906 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb38a5-4a35-449d-bd7c-23cf23fa79e8 service swift] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:45.924211 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:45.924835 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:45.925112 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:45.925382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] PATH_INFO: `/v3/projects/564b15ef544147a383f39bbfda4a1ecd` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:45.926840 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=564b15ef544147a383f39bbfda4a1ecd)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:45.928779 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:46.040722 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564b15ef544147a383f39bbfda4a1ecd', 'request_id': 'req-41d50d64-e726-4a8f-8b7b-b17819c4c655'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.052020 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564b15ef544147a383f39bbfda4a1ecd', 'request_id': 'req-41d50d64-e726-4a8f-8b7b-b17819c4c655'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.058761 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564b15ef544147a383f39bbfda4a1ecd', 'request_id': 'req-41d50d64-e726-4a8f-8b7b-b17819c4c655'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.147960 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] The token cache is being invalidate because project 564b15ef544147a383f39bbfda4a1ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:46.148395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.149029 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.150582 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1252/2525] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:45 2022] DELETE /identity/v3/projects/564b15ef544147a383f39bbfda4a1ecd => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:46.154725 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:46.157888 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:46.172586 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543cd36-2b7e-4fe5-a15e-26eee5d596ef admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:46.235330 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:46.235993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:46.236257 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:46.236525 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] PATH_INFO: `/v3/users/432c42f83107420db971b035047b52f2` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:46.237859 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=432c42f83107420db971b035047b52f2)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:46.239986 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:46.544378 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '432c42f83107420db971b035047b52f2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.545029 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '432c42f83107420db971b035047b52f2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.837964 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '432c42f83107420db971b035047b52f2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.838563 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '432c42f83107420db971b035047b52f2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.846572 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '432c42f83107420db971b035047b52f2'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:46.891022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1274/2526] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:46 2022] DELETE /identity/v3/users/432c42f83107420db971b035047b52f2 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:47.171780 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:47.173395 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:47.190088 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d50d64-e726-4a8f-8b7b-b17819c4c655 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:47.238591 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:47.250691 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:47.251325 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:47.251617 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:47.251915 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] PATH_INFO: `/v3/projects/14332372c0304c00b26b8b97ff95f816` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:47.253062 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=14332372c0304c00b26b8b97ff95f816)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:47.255056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:47.487685 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14332372c0304c00b26b8b97ff95f816', 'request_id': 'req-a57a5b8b-6126-4be7-9df8-633e07788d9e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:47.493298 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14332372c0304c00b26b8b97ff95f816', 'request_id': 'req-a57a5b8b-6126-4be7-9df8-633e07788d9e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:47.500709 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14332372c0304c00b26b8b97ff95f816', 'request_id': 'req-a57a5b8b-6126-4be7-9df8-633e07788d9e'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:47.759367 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] The token cache is being invalidate because project 14332372c0304c00b26b8b97ff95f816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:47.759585 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:47.760231 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-a57a5b8b-6126-4be7-9df8-633e07788d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:47.761849 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1253/2527] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:47 2022] DELETE /identity/v3/projects/14332372c0304c00b26b8b97ff95f816 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:48.136782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:48.143532 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:48.152935 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a7de-f1a1-4f8c-931f-fd1d2486f45b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:48.174375 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:48.213418 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:48.214049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:48.214362 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:48.214640 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] PATH_INFO: `/v3/users/72b52dd0fa2848e6ab2b64151f4d54fd` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:48.215738 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b52dd0fa2848e6ab2b64151f4d54fd)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:48.217681 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:48.435244 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b52dd0fa2848e6ab2b64151f4d54fd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:48.435925 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b52dd0fa2848e6ab2b64151f4d54fd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:48.529774 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b52dd0fa2848e6ab2b64151f4d54fd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:48.530470 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b52dd0fa2848e6ab2b64151f4d54fd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:48.537744 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b52dd0fa2848e6ab2b64151f4d54fd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:48.555666 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1275/2528] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:48 2022] DELETE /identity/v3/users/72b52dd0fa2848e6ab2b64151f4d54fd => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:48.560204 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:48.560698 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:48.560799 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:48.580888 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:48.582076 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:48.639219 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1254/2529] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:48.650023 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:48.651401 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:48.709220 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a749d0-ae0c-4c7a-9f76-e482fced3bb2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:48.712680 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:48.713276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:48.713567 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:48.713814 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:48.722910 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:48.724370 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:48.734442 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1276/2530] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:48.994650 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:48.996091 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:49.026465 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfa264-a1f9-4887-acbe-72e35d18c918 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:49.036412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:49.036412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:49.036412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:49.036412 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] PATH_INFO: `/v3/projects/dde3823afe64468db10f0141d8735a7e` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:49.036825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde3823afe64468db10f0141d8735a7e)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:49.036825 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:49.164645 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde3823afe64468db10f0141d8735a7e', 'request_id': 'req-906e5572-6600-4b0a-a1d0-c1f11a43b955'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:49.170058 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde3823afe64468db10f0141d8735a7e', 'request_id': 'req-906e5572-6600-4b0a-a1d0-c1f11a43b955'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:49.177458 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde3823afe64468db10f0141d8735a7e', 'request_id': 'req-906e5572-6600-4b0a-a1d0-c1f11a43b955'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:49.278195 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] The token cache is being invalidate because project dde3823afe64468db10f0141d8735a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:49.278414 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:49.279060 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-906e5572-6600-4b0a-a1d0-c1f11a43b955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:49.281239 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1255/2531] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:48 2022] DELETE /identity/v3/projects/dde3823afe64468db10f0141d8735a7e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:52.955046 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:52.956498 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:52.969466 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa926526-aa8e-4bfe-8fa7-11e1f9e57581 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:53.013112 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:53.028403 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:53.029056 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:53.029335 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:53.029611 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] PATH_INFO: `/v3/users/6ebf625256b34f27b1b2a29e2ba0cc4d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:53.031408 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ebf625256b34f27b1b2a29e2ba0cc4d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:53.033377 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:53.252115 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ebf625256b34f27b1b2a29e2ba0cc4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:53.252806 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ebf625256b34f27b1b2a29e2ba0cc4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:53.352290 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ebf625256b34f27b1b2a29e2ba0cc4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:53.352993 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ebf625256b34f27b1b2a29e2ba0cc4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:53.361231 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ebf625256b34f27b1b2a29e2ba0cc4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:53.380979 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1277/2532] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:52 2022] DELETE /identity/v3/users/6ebf625256b34f27b1b2a29e2ba0cc4d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:53.385432 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:53.385924 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:53.386025 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:53.393938 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:53.407720 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:53.408951 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:53.469666 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1256/2533] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:53.480543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:53.481929 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:53.541388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4a315-1e2e-4cb4-8fd6-fdae7c7af7ab admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:53.544903 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd0d2-a24b-4eb9-9dc8-3b0302f9c935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:53.545505 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd0d2-a24b-4eb9-9dc8-3b0302f9c935 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:53.545782 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd0d2-a24b-4eb9-9dc8-3b0302f9c935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:53.546663 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd0d2-a24b-4eb9-9dc8-3b0302f9c935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:53.555855 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd0d2-a24b-4eb9-9dc8-3b0302f9c935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:53.557278 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd0d2-a24b-4eb9-9dc8-3b0302f9c935 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:53.569205 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1278/2534] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:54.023300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:54.024897 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:54.040860 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26713d-3348-4c34-a17e-12559ee77a3d None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:54.048407 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:54.049061 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.049347 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.049620 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] PATH_INFO: `/v3/projects/0a4baf33753b4691b5463392f5b2ab59` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.050758 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4baf33753b4691b5463392f5b2ab59)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:54.052769 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:54.175719 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4baf33753b4691b5463392f5b2ab59', 'request_id': 'req-c4810aae-dbfd-4afe-a90a-b6effe91cf94'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:54.181420 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4baf33753b4691b5463392f5b2ab59', 'request_id': 'req-c4810aae-dbfd-4afe-a90a-b6effe91cf94'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:54.187804 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4baf33753b4691b5463392f5b2ab59', 'request_id': 'req-c4810aae-dbfd-4afe-a90a-b6effe91cf94'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:54.278115 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] The token cache is being invalidate because project 0a4baf33753b4691b5463392f5b2ab59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:54.278433 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:54.279085 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-c4810aae-dbfd-4afe-a90a-b6effe91cf94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:54.281317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1257/2535] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:54 2022] DELETE /identity/v3/projects/0a4baf33753b4691b5463392f5b2ab59 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:54.286588 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a87062-31a0-48b3-870a-4aed736eaf3a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.287052 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a87062-31a0-48b3-870a-4aed736eaf3a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.287168 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a87062-31a0-48b3-870a-4aed736eaf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.307699 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-90a87062-31a0-48b3-870a-4aed736eaf3a None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:54.308946 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-90a87062-31a0-48b3-870a-4aed736eaf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:54.369432 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1279/2536] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:54.377515 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e300735-9ba8-4719-a854-d7d205b19968 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.377796 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e300735-9ba8-4719-a854-d7d205b19968 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.378066 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e300735-9ba8-4719-a854-d7d205b19968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.398671 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-1e300735-9ba8-4719-a854-d7d205b19968 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:54.399895 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-1e300735-9ba8-4719-a854-d7d205b19968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:54.408383 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1258/2537] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:54.412499 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a87062-31a0-48b3-870a-4aed736eaf3a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:54.421904 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a87062-31a0-48b3-870a-4aed736eaf3a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:54.426585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:54.427113 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.427388 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.427654 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.428325 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:15:54.428624 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:54.430195 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:54.431631 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:54.442242 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1280/2538] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 26 11:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:15:54.448995 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e300735-9ba8-4719-a854-d7d205b19968 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:54.458955 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e300735-9ba8-4719-a854-d7d205b19968 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:54.463674 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:54.464299 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.464574 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.464842 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] PATH_INFO: `/v3/projects` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.465470 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:54.467427 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:54.554610 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1259/2539] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 26 11:15:54 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:54.558997 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:54.569584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2559586-f9c8-4418-a1f8-6b8dc9fd1ade admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:54.626560 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:54.627167 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.627447 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.627719 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.628385 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:54.630299 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:54.769889 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1281/2540] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 26 11:15:54 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:15:54.774056 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:54.784586 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e84f8-8b09-4746-a901-621502f92073 admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:54.789071 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:54.789661 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.789936 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.790203 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.790867 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:54.792491 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:54.802815 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1260/2541] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:15:54.806742 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:54.817593 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf56dfc-4935-464a-a054-d61d9329f1b0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:54.822386 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d8a0e-5938-4f2f-ad1c-60d8ece55033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:54.822975 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d8a0e-5938-4f2f-ad1c-60d8ece55033 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.823252 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d8a0e-5938-4f2f-ad1c-60d8ece55033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.823518 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d8a0e-5938-4f2f-ad1c-60d8ece55033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.824188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d8a0e-5938-4f2f-ad1c-60d8ece55033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:54.825845 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d8a0e-5938-4f2f-ad1c-60d8ece55033 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:54.837096 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1282/2542] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 26 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:15:54.841263 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:54.851681 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8b58-7e02-4e67-ad7c-eda549c40f8a admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:54.859374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:54.859374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.859374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.859374 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] PATH_INFO: `/v3/projects/cc0e8851b2f7441e8416d8760cc121aa/users/cdd3753e10004af6bd8214b880922a09/roles/b109a364aa914a98aee2e78986065def` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:54.873074 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0e8851b2f7441e8416d8760cc121aa, user_id=cdd3753e10004af6bd8214b880922a09, role_id=b109a364aa914a98aee2e78986065def)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:54.876534 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:54.970300 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1261/2543] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 26 11:15:54 2022] PUT /identity/v3/projects/cc0e8851b2f7441e8416d8760cc121aa/users/cdd3753e10004af6bd8214b880922a09/roles/b109a364aa914a98aee2e78986065def => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:15:54.980775 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeee11-3e64-41a5-aa73-6ec66ba9ca07 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:54.981062 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeee11-3e64-41a5-aa73-6ec66ba9ca07 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:54.981390 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeee11-3e64-41a5-aa73-6ec66ba9ca07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:55.013049 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-afeeee11-3e64-41a5-aa73-6ec66ba9ca07 None None] MFA Rules not processed for user `cdd3753e10004af6bd8214b880922a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:55.014275 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-afeeee11-3e64-41a5-aa73-6ec66ba9ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:55.082887 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1283/2544] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:55.094405 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:55.096466 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:55.126489 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:55.160557 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957105b-2072-4018-a71f-4aed6cb23d7f admin admin] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:55.164513 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5f762-b3fc-4c9f-8e90-22be78b5fc51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b474a596cf42efb5e0e5a2a179735e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:55.165476 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5f762-b3fc-4c9f-8e90-22be78b5fc51 service cinder] REQUEST_METHOD: `GET` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:55.165801 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5f762-b3fc-4c9f-8e90-22be78b5fc51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:55.166075 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5f762-b3fc-4c9f-8e90-22be78b5fc51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:55.174921 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5f762-b3fc-4c9f-8e90-22be78b5fc51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:55.177207 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5f762-b3fc-4c9f-8e90-22be78b5fc51 service cinder] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:55.186570 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1262/2545] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:55.212924 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeeee11-3e64-41a5-aa73-6ec66ba9ca07 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:55.275658 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeeee11-3e64-41a5-aa73-6ec66ba9ca07 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:55.281079 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:55.282276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:55.282276 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:55.282475 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] PATH_INFO: `/v3/users/cdd3753e10004af6bd8214b880922a09` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:55.283558 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd3753e10004af6bd8214b880922a09)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:55.285516 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:55.287908 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:15:55.502932 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd3753e10004af6bd8214b880922a09'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:55.503585 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd3753e10004af6bd8214b880922a09'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:55.633076 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd3753e10004af6bd8214b880922a09'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:55.633717 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd3753e10004af6bd8214b880922a09'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:55.641867 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd3753e10004af6bd8214b880922a09'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:55.660179 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1284/2546] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 26 11:15:55 2022] DELETE /identity/v3/users/cdd3753e10004af6bd8214b880922a09 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:15:55.664770 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] REQUEST_METHOD: `POST` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:55.665269 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:55.665382 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:55.685362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.auth.core [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] MFA Rules not processed for user `36c1a5e80256403c902f09eda7736a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81608) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:15:55.686568 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:55.745101 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1263/2547] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 26 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:15:55.754723 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:55.756188 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:55.813584 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0d34a-3980-442f-8aed-abe41032bd7f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:55.817081 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0cb1d976784ea6af783fd193ddac20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:55.817683 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:55.820668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:55.820668 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:55.827011 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:55.829022 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:55.838273 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1285/2548] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:15:56.378542 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] Authenticating user token {{(pid=81608) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:15:56.380260 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.fernet_utils [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81608) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:15:56.393640 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324564e3-7b8a-4ac8-ad4a-22c1e2986594 None None] Validating token access rules against request {{(pid=81608) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:15:56.398437 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c1a5e80256403c902f09eda7736a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00b7a3900004d669310a34e206f2fc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81608) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:15:56.399362 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:15:56.399853 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:15:56.400366 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] PATH_INFO: `/v3/projects/cc0e8851b2f7441e8416d8760cc121aa` {{(pid=81608) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:15:56.402434 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0e8851b2f7441e8416d8760cc121aa)` {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:15:56.403982 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] RBAC: Authorization granted {{(pid=81608) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:15:56.526317 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0e8851b2f7441e8416d8760cc121aa', 'request_id': 'req-6281f9cd-d987-46de-aa6e-34556ecb8368'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:56.532518 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0e8851b2f7441e8416d8760cc121aa', 'request_id': 'req-6281f9cd-d987-46de-aa6e-34556ecb8368'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:56.540734 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0e8851b2f7441e8416d8760cc121aa', 'request_id': 'req-6281f9cd-d987-46de-aa6e-34556ecb8368'} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:56.637837 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] The token cache is being invalidate because project cc0e8851b2f7441e8416d8760cc121aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81608) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:15:56.638136 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:56.638784 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG keystone.notifications [None req-6281f9cd-d987-46de-aa6e-34556ecb8368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81608) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:15:56.641169 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: [pid: 81608|app: 0|req: 1264/2549] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 26 11:15:56 2022] DELETE /identity/v3/projects/cc0e8851b2f7441e8416d8760cc121aa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:16:06.032590 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:16:06.634657 dv-cinder-862679-1-iscsi devstack@keystone.service[81608]: DEBUG dbcounter [-] [81608] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81608) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:35.173024 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.174416 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.311376 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac70d9-2882-4b72-973a-cdd2c0a4cdc1 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.369945 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3f4a6-e3b2-4e86-8a57-11e2bbead3a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c154983aff14d9895fb4b542db61e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5968f35bae3d4643bc199a16ab96770b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.370543 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3f4a6-e3b2-4e86-8a57-11e2bbead3a6 service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.370819 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3f4a6-e3b2-4e86-8a57-11e2bbead3a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.371087 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3f4a6-e3b2-4e86-8a57-11e2bbead3a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.372812 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-66b3f4a6-e3b2-4e86-8a57-11e2bbead3a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.384158 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3f4a6-e3b2-4e86-8a57-11e2bbead3a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.385716 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3f4a6-e3b2-4e86-8a57-11e2bbead3a6 service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.396095 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1286/2550] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:45.391582 dv-cinder-862679-1-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}